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Executive Sumary 



This study evaluated the natlcnal network of Technical Assistance 
Centers (TACs)i which provides technical assistance in evaluation and 
program improvement to state a*ad local educational agencies (SEAs and LEAs) 
responsible for implementing programs under Chapter 1 of the Education 
Consolidation and Improvement Act (ECIA). The U.S. Departaient of Education 
(ED) currently operates four TACs, which each provide assistrance in a 
specified region of the country. The TACs are funded at an overall level of 
$3.6 million a year, down from a high of $8.5 million in 1980-81. 



TAC Services and Operations 

The TACs were established in 1976. to assist SEAs and LEAs in 
implementing the project evaluation requirements of Title I of the 
Elementary and Secondary Education Act. They continue to provide the 
following types of assistance in evaluating compensatory education programs: 

• Much of the TACs' current assistance in evaluation consists of 
answering questions on testing « including questions on aligning 
tests and curriculumi test selection, scoring, and report 
preparation. 

• In response to growing demand, TACs provide in*depth assistance in 
implementing the Chapter 1 suscained effects requirements, 
developing microcomputer data bases, and interpreting evaluation 
data. 

• As states increase their student assessment activities, TACs are 
being asked to advise and assist in coordinating Chapter 1 testing 
activities with state assessment programs. 

TACs also provide wide-ranging assistance in improving the quality and 

effectiveness of Chapter 1 instructional services, as follows: 

• Program improvement services typically include assistance with 
needs assessment (using locally developed evaluation data) and 
with designing and implementing improvement plans. 



• TAG assistance in prograa Improvoment includes help with 
interpreting data, rethinking curriculuffl^ planning change, 
conducting staff development, involving parents, coordinating with 
and influencing the regular instructional program, and de^'ieloping 
strategies for continuing self-assef^sment. 

• Assistance in these areas generally culminates in a locally 
generated Chapter 1 improvement plan that includes steps aimed at 
developing project and school characteristics associated with 
instructional effectiveness. 

• TACs generally deliver these services through workshop series or 
direct consultations in LEAs. 

We found that, despite variations across TACs, they have generally 
selected technical assistance strategies that are consistent with their 
program goals. Moreover, TACs' administrative frameworks facilitate the 
delivery of the intended services. Our analysis indicates that the costs of 
TAC services are generally reasonable. 

The major exceptions to this picture of a coherent, efficient system of 
technical assistance are: 

• TACs continue to provide some services (e.g., explaining Chapter 1 
evaluation requirements, helping prepare SEA reports) that SEAs 
should be able to handle on their own. 

• TAC assistance in program improvement does not consistently 
involve SEAs in ways that help them improve their own 
capabilities. 

• The availability of TAC assistance in program improvement is 
creating demands for services that may exceed the current system's 
capacity to- fulfill. 

• Large states are not receiving an equitable share of TAC ^service, 
due to the high floor of service necessary for each state and the 
low aggregate level of service. 

• TAC offices have relatively little contact with each other and 
thtis miss opportunities to share materials and approaches. 

o TAC staff receive little or no training in the provision of 
technical assistance. 



ii6 



The Users of TAG Assistance 



'Current data on SEA and LEA staff resources for implementing Chapter 1 
indicate significant levels of need for a^rsistance in evaluating and 
improving program services. These needs are particularly serious in light 
of tthe program's regulatory tradition » in which substantial amoxmts of staff 
time and attention go to maintaining compliance with legal requirements. 
Under these conditions » the expertise and availability of the TACs cannot 
help but make a welcome contribution* 

Looking at the match between what TACs offer and what SEAs and 
districts want» we identified three different roles that TACs fuX>Eill: 

• TAC staff act as a reference service for virtually all their state 
and local clients. This is their least demanding role in terms of 
time and skills » but it meets an important need for information. 

• TACs serve as -extensions of SEA staff. Because the overall 
numbers of SEA staff who specialize in Chapter 1 evaluation or 
program content are so low (averaging about half a full-time 
equivalent staff meinber in each area per state) , skilled help from 
the TACs makes a real difference. 

• TACs act as capacity builders when they and local districts commit 
reasonable amounts of time to an intervention such as a workshop 
series* In addition to the skills they teach » the TACs help build 
Chapter 1 instructional capacity simply by serving as advocates 
for improvement. 

The Effects of TAC Services 

Despite differences across TACs, we found that in general they are 
achieving results commensurate with their efforts in evaluation and program 
improvement assistance. The most important of their results may be the 
Interest and enthusiasm they generate among. local Chapter 1 personnel. 
Interactions with TACs help local staff see new possibilities in the 
instructional services they provide and the results that their students are 
capable of achieving. In addition, the following effects are also evident: 

ill 
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• State and local users are satisfied with TAG services. 

• TAG assistance has resulted In the successful adoption and 
Implementation of Chapter I's evaluation and reporting system. 

• The TAGS have halped states Improve th\ quality of their Chapter 1 
data. 

• They have helped familiarize LEAs with research findings on 
effective schools and classrooms. 

• Because program Improvement assistance draws direct connections 
between evaluation results and Chapter 1 programs^ TAG help In 
this area has Increased local Interest In evaluation Issues. 

• TAG assistance In program Improvement has also prompted greater 
coordination between Chapter 1 and regular Instruction. 

• Finally, the TAGs have helped communicate ED priorities In program 
Improvement and evaltiatlon. 

Given the very large number of Chapter 1 LEAs and the small size of the 

TAG program, the positive effects of TAG services will be limited to only a 

few local recipients unless SEAs can be enlisted to participate more 

meaningfully in program Improvement assistance* SEA Chapter 1 offices are 

the logical entitles for this ^>?ork because they know the Chapter 1 program 

and the characteristics of the projects in their states. What they 

sometimes lack is expertise in program improvement and technical assistance. 

To help them acquire that expertise, they need to learn from the TAGs and 

the TAGs xieed to encourage them to leazm. 

The TACs and ED 

Staff availability and contracting procedures constrain ED's 
administration of the TAGs in ways that are not easily changed. Even so, in 
anticipating shifts in the demands made on the TAG program, we found that 
three results of the current relationship between ED and the TAGs are 
particularly important. First, current TAG requirements and incentives 
place low priority on the development of expertise or materials specially 
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tailored to the needs of TAC users. Second, few Incentives or opportunities 
exist to encourage' information- sharing and collaboration across TACs. 
Third, current federal monitoring procedures dp not encourage TACs to engage 
in the kinds of in-depth consultation and sustained involvement with local 
staff that are most likely to produce lasting improvement in Chapter 1 
services • 



Alternatives for Change in the TAG Program 

In designing specifications for the next TACs, we suggest that ED 
redirect the TAC mission towards increasing SEAs^ capacity to promote and 
Implement Chapter 1 Improvement , This role would not preclude current TAC 
services, but it would cause TACs and SEAs to place more emphasis on TAC 
efforts tio build SEA capacity as assistance providers and advocates for 
improvement* To supplement this shift, we suggest the following additional 
alternatives for change in the TAC program: 

Possible Changes In the Structure of the TAC Program 

• Assign special areas of expertise and responsibility to some TACs 
(e*g*» parent involvement, education of migrant students). 

• Establish minimum levels of effort for TAC offices. 
Possible Changes tn the General Responsibilities of the TACs 

• Require TACs to evaluate their technical assistance services and 
to use evaluation data to improve their service capacities. 

• Increase TAC capacity to assist in developing student- and school- 
level data bases, 

• Increase TAC capacity to assist in coordinating Chapter 1 
evaluation and program improvement with state testing and 
improvement initiatives. 

• Conduct a series of TAC seminars on technical assistance issues. 

• Ftd^lish a national newsletter on Chapter 1 program improvement and 
evaluation. 
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Possible Changes in ED Administration of the TACs 

• Implement new reporting requirements. 

• Increase feedback to TACs on their performance, 
e. Allow TACs greater latitude in staffing. 
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1. Introduction 



For the past twelve years, a national network of Technical Assistance 
Centersf (TACs) has provided assistance In evaluation and program Improvement 
to agencies responsible for administering federal compensatory education 
programs. Originally focused on assisting the Implementation of required 
evaluation procedures, the mission of the TACs has changed to Include 
assistance In improving the Instructional programs supported by federal 
compensatory funds. As the U.S. Department of Education (ED) plans for 
changes In the compensatory education program, It will need to decide what 
TAC program features sUould be modified or retained. This report is 
intended to provide information and analysis for those decisions. 

Origin and Development of the TAC Program of Technical Assistance 

Program and project evaluation has been a central part of the federal 
compensatory education program since the 1963 enactment of Title I of the 
Elementary and Secondary Education Act (ESEA) , when Senator Robert F. 
Kennedy added the requirement that local educational agencies (LEAs) 
evaluate the effectiveness of their Title I projects in serving 
educationally deprived children. To increase local uniformity in 
implementing this mandate, the 1974 amendments to Title I required the 
Commissioner of Education to develop and implement **standards for evaluation 
of program or project effectiveness in achieving the objectives" of Title I, 
including **models for all programs conducted** under the program. The 1974 
amendments also required the Commissioner to provide technical assistance to 
state education agencies (SEAs) **to enable them to assist LEAs in 



development and application of a systematic evaluation, of programs in 
accordance with the models*** 

In response to the legislative mandate, the Office of Education (OE) 
established ten TACs in 1976, funding them under Title I's evaluation set- 
aside* Under these contracts, the primary TAG objective was to assist SEAs 
and LEAs in adopting and implementing the newly developed evaluation models, 
in order to generate data on the achievement changes of Title I students. 
As educational agencies became increasingly familiar with the models, OE 
encouraged the TACs to assist in improving the quality of the Title I data. 

Chapter 1 of the Education Consolidation and Improvement Act (ECIA) , 
which replaced Title I, made several changes in the program's evaluation 
requirements, including removal of provisions requiring (1) SEAs and LEAs to 
implement the Title I evaluation models and (2) federal support of technical 
assistance in Chapter 1 evaluation* It added a new provision, however, 
authorizing the Secretary of Education to **provide technical assistance 
• . . to promote the development and implementation of effective 
instructional programs* ***** Although ED has continued to support the 
TACs under ECIA, it has reduced their number to four and directed them to 
provide assistance in improving Chapter 1 programs* 

Current funding for the TACs is about $3*6 million a year, down from a 
high of $8*5 million in 1980-81* Funds are allocated about equally across 
the four TACs* The TAC regions the current TAC contractors, and their 
locations are shown in Figure 1* 

This evaluation of the TACs is the third that ED has sponsored* the 
earlier two were: 
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Figure I 



TAG REGIONS 



REGION 2: Advanced technology, Hampton, N«w Hampshire 

Indianapolis, Indiana ^^c. Woshtngfon. D.C. 

Field Office: Research and Training Associates 
Overland Park, Kansas 




REGION 1: Educational Testing Service 

Princeton, New Jersey 
Field Office: RMC Research Corporation 



nEGION 4; NWREL 
Portland. Oregon 

Field Office: nwrel 
Denver, Colorado 



REGION 3: Educational Testing oervice 

Allanla. Georgia 
Reld Office: Poweli Associates, Inc. 
AusMn, Texas 



Original version prepared by Northwest Regional Educational Laboratory (NWREL) 



• 1978/79 Performance Review of the Technical Assistance Program 



Conducted by an OE-established panel, the review concluded that 
the technical assistance program was "working and working well" 
(Hillman, Paisley, Rogers, Sanders, & Womer, 1979, p. 57). It made 
a number of recommendations for improvement (e.g., longer contract 
periods, more emphasis on using evaluations to improve programs, 
more uniform reporting procedures), which OE implemented in the 
subsequent round of TAC contracts. 

• 1982 Evaluation of the TACs 

Conducted as part of a larger assessment of the overall Title I 
Evaluation and Reporting System (TIERS- -Reisner, Alkins, Boruch, 
Linn, & Millman, 1982), the study found that: 

The content of TAC assistance had "shifted from implementing 
the TIERS models to improving the quality and utility of 
evaluation data" (p. 43). 

The magnitude and content of services varied across states 
and TACs. 



"The amount of field service is low relative to the money 
expended" (p. 43). 

"The TAC clients are well satisfied with the TAC services and 
want them to continue" (p. 43). 

The evaluation presented in this report addresses many of the issues 

raised in the earlier studies. It is intended to provide information for 

TAC program changes linked to the reauthorization of Chapter 1 and a 

competition to select new TAC contractors. 



Purposes and Methods of This Evaluation 

The purposes of this evaluation were to (1) describe TAC operations, 
especially current activities; (2) assess the utility of TAC assistance to 
SEAs and LEAs; and (3) examine future needs for technical assistance. The 
study team was not asked to evaluate the performance of individual TAC 
contractors . 

We addressed the evaluation's first purpose through on-site interviews 
and the inspection of materials at each of the TAC headquarters and field 
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offices. The site visits utilized Interview guides and debriefing forms 
designed to permit conclusions across TACs. 

We addressed the second purpose through telephone interviews with 
Chapter 1 coordinators and, in some cases, evaluators in each of nine SEAs 
and 14 LEAs. SEAs selected for the study Included one SEA served by each 
TAC office. The sample of SEAs included states of differing size and SEA 
philosophy. Our LEA sample Included one LEA in each of the sample SEAs; 
these LEAs varied by enrollment size. In addition, we interviewed Chapter 1 
coordinators in five other LEAs that had received intensive TAC assistance 
in improving their Chapter 1 services. We did not tell the TACs which SEAs 
or LEAs were In our samples, nor did we tell the SEAs which LEAs we had 
selected to interview. Our telephone interviews included questions 
regarding technical assistance needs (past, current, and future), sources of 
technical assistance, receipt of TAC services, and strengths and weaknesses 
of TAC assistance. 

To learn more about future needs for Chapter 1 technical assistance, we 
also reviewed the relevant House and Senate reauthorization bills and the 
resulting act, and we interviewed Washington -based analysts familiar with 
reauthorization plans and current TAC capacities. 

This report presents the results of our evaluation. Chapter 2 
describes TAC services and operations. Chapter 3 explains the state and 
local contexts in which TACs provide services, especially state and local 
activities in program evaluation and Improvement. Chapter 4 summarizes the 
effects of TAC services. Chapter 5 discusses the relationship between the 
TACs and ED. The final chapter presents alternatives for Improvement in TAC 
services, based on current operations and our analysis of future needs. 

5 
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2« TAC Services and Operations 

« 

This chapter describes the technical assistance that TACs provide and 
the administrative activities that they carry out to support the. delivery of 
technical assistance. It looks at TAC services and operations primarily 
from the viewpoint of the TACs themselves. Subsequent chapters describe 
pressures exerted on the TACs by state and local users of TAC services and 
by ED. 

TAC Services to SEAs and LEAs 

Most TAC staff view evaluation and program imjprovement as interwoven 
activities. From their perspective, the goal of evaluation is to produce 
valid and reliable data that can be used to assess program strengths and 
weaknesses and guide improvement efforts. Because of this link, the 
distinction between assistance in evaluation and program improvement is 
often blurred. Even so, the Chapter 1 statute and the TACs' contractual 
requirements distinguish between the two kinds of assistance, as we do 
below. 

Assistance in Chapter 1 Evaluation 

Most TAC assistance in evaluation consists of short-term technical help 

on specific problems raised by SEAs and LEAs. Depending on the problem and 

the client, TAC staff generally send out prepared materials, answer 

questions on the telephone, or conduct workshops for state or local staff. 

The demand for this assistance has gradually decreased in recent years for 

several reasons, including the following: 

• State and local Chapter 1 coordinators and evaluators understand 
Chapter 1 evaluation and reporting requirements better than they 
did in the early days of TIERS. One state coordinator whom we 
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interviewed said, for example, that implementation of TIERS is now 
"common practice.** 

• Beceu^Q SEAs know fewer TAG staff are available to help now than 
several years ago, they are more likely to cry and resolve minor 
problems on their own- -and they characterize many evaluation 
problems as minor. 

• Based on our inten^iews, SEAs appear to have relatively greater 
need for assistance in program improvement than in evaluation. 
Requests for improvement help have thus crowded out many requests 
for assistance in evalxxation. The requests for help in improving 
programs often involve evaltxation (e.g., use of sustained effects 
analyses to identify areas needing improvement), however. 

• SEAs are aware that TACs must spend half their time on program 
improvement, and so they have shifted their requests accordingly. 
With few exceptions, both districts and states say that they 
welcome the emphasis on improvement. 

Assistance in understan ding TIERS . The kinds of evaluation problems 
for which SEA and LEA staff request help have shifted over time. Initially, 
explaining TIERS and helping SEAs and LEAs develop procedures for collecting 
and reporting achievement data were the main focus of TAG assistance. 
Although explaining TIERS is still necessai^, due mostly to turnover of 
Chapter 1 staff, it is a relatively infrequent activity now and mainly 
consists of sending materials in response to requests from new staff and 
conducting occasional workshops as part of state or regional conferences. 
As one SEA respondent said, "We no longar heed help on models because local 
and state staff have become proficient, except for sustained effects where 
help is still needed." Because few states use sampling any more, TAGs are 
rarely asked to help SEAs in this area. 

Help to SEAs in preparing evaluation reports. This kind of assistance 
is no longer a major activity for TAG staff. Although TAGs still assist 
some SEAs with reports to ED and their LEAs, they report that they generally 
have too little time to do mucl» report preparation. SEAs that use 
consultants to prepare their reports or that have set up good data 
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:management systems (often with TAG help) do not request assistance with 
their reports. Relatively few SEAs prepare reports for their LEAs. 

Advice on selecting and scoring tests . Based on our SEA and LEA 
intervifivs, help with testing issues is by far the most frequently requested 
form of evaltuttion assistance. This reflects a constant but low level of 
demand for information, although less now than in the past, according to TAG 
staff. TAG staff do not recommend specific tests, but they do provide 
general information about how to choose a test and the characteristics of 
available tests. 

Host TAG assistance on testing issues addresses specific questions that 
SEAs or LEAs convey by telephone; for example, an LEA may have questions 
about the norms for a newly published version of a commercial test. TAGs 
often answer these questions by mailing out materials prepared by either a 
TAG, the TAGs' own Test Information Genter, or the Test Genter at the 
Northwest Regional Educational Laboratory. TAGs also sometimes provide 
workshops on testing issues; for example, one state obtained assistance on 
test selection as part of its effort to align its Ghapter 1 curricula with 
its assessment activities. 

Assistance in setting up computerized data bases. Over the last few 
years, many SEAs and large LEAs have turned to TAGs for help in setting up 
computerized data bases to manage Ghapter 1 participation and achievement 
data. The requirements for evaluating Ghapter I's sustained effects and the 
interest in identifying low performing schools have increased the demand for 
these services. In addition to other benefits, TAG and SEA staff view the 
automation of data entry and aggregation as one of the best ways of 
improving the quality of evaluation data. 
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TAC assistance in this area has taken several forms* In addition to 
helping states create data systems, TAC staffs have also developed templates 
for commercial data hash software, permitting SEAs and LEAs to use 
relatively inexpensive software to track student-level data. One TAC has 
developed a program that automatically checks for common errors during data 
entry, such as scores that are out of range. SEAs and large LEAs are the 
main recipients of TAC help in setting up large automated data management 
systems, while small LEAs have received the most help in adapting software 
for data entry on microcomputers. Some SEAs with automated systems have 
obtained TAC help in upgrading their systems, as analysis demands increase. 
Three of the four TACs report more demand for assistance in this area than 
they can meet. 

pelp in measuring sustained effects . Another evaluation service that 
TACs provide is help in implementing the requirements to evaluate Chapter 
I's sustained effects. The TACs have provided introductory workshops on 
implementing this requirement, followed in some instances by individual 
consultations with districts. LEAs have asked for this personalized help in 
order to integrate the required procedures with their regular collection and 
analysis of Chapter 1 participation and achievement data. 

Assistance In coordinating Chapter 1 and state assessment > TAC staff 
see several benefits in helping to design procedures for coordinating 
Chapter 1 evaluation and state assessment activities. One benefit is that 
state testing often occurs imder more controlled conditions, thus producing 
more accurate results than Chapter 1 testing. (According to a TAC staff 
member. Chapter 1 testing often occurs in the back of a busy classroom 
during test periods that are shortened or lengthened to fit the class 
period.) In addition, using the same test for Chapter 1 and state 
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assessnent purposes reduces the testing burden on students. However » TACs 



and state respondents report several barriers to such coordination: 

• Alaost all states with their own testing programs administer 
tests in only a few grades* -every other grade at most. Because 
Chapter 1 students aeust be tested at 12*month intervals, 
consecutive grades must be tested each year for Chapter 1 
p\irposes • 

• Some state assessment programs select samples of students to be 
tested, in order to generate scores reflecting school or district 
performance. The statf) ti^st samples are usually not designed to 
reflect the performance of ' Chapter 1 students. 

• Many state assessment instruments are tests of minimal 
competencies and therefore cannot measure growth in academic 
achievement, as required by TIERS. 

• Turf issues between Chapter 1 and state assessment offices 
sometimes create political impediments to test coordination. 

TAC assistance in coordinating test programs has taken several forms. 

One state Chapter 1 coordinator said, **The TAC has given vs a lot of 

assistance" in resolving ongoing coordination problems in the state's three- 

year effort to shift Chapter 1 testing to the state's assessment program. 

In another state, the Chapter 1 coordinator reported that the TAC helped 

develop an LEA manual demonstrating how to xise the state test for Chapter 1 

purposes. 

Asststrance to Chanter 1 state-operated migrant education and neglected 
or delinquent programs , TAC directors reported varying experiences in 
providing evaluation assistance to these programs, although all provide more 
assistance to migrant education than to neglected or delinquent projects. 
As5istance to migrant projects takes several forms, as des^^ribed below: 

• TACs help some migrant projects implcraent TIERS, usually by 
explaining TIERS requirements, helping in the selection of tests, 
or conducting related activities. These projects mainly serve 
migrant students who do not move during the school year (i.e., 
those who are "formerly migratory** and students who move only 
during the summer months). 
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• tACs help some o ler algrant proj'^cts lapleaent evaluations that 
d6 not rely on pie- and post* tes\ .ng. These projects serve 
students vho are not enrolled long enough in the school year to be 
tested twice* 

• TACs also help soiie sui:?ier*only projects conduct evaluaticv^s 
designed to give snapshots of students' achievement levels* 

• In several states, TAG staff have helped in the development of 
migrant education data bases Intended to supplement and expand the 
data routinely collected for the Migrant Student Record Transfer 
System. 

• TAG staff helped one large SEA ti99B9S the dro*^out problem in 
migrant education. 

One TAG director voiced fairly low expectations for the evaluation of 
Chapter 1 migrant programs, **There are a lot of requests for assistance; 
we're forttinate if we can gee them [migrant program directors] to take 
respot:9ibility for evaluation and sustained affects.** 

Directors reported that the TAG's relationship with the state migrant 
office affects the provision of assistance. Because TAGs work mainly with 
state Chapter 1 offices, they tend to have little contact with the migrant 
program in states where it is implemented by a separate migrant education 
office. The only exceptions are the TAGs t!tat have made special efforts in 
migrant education; their outreach efforts to migrant offices have resulted 
in requests for evaluation assistance. One TAG reported that it had gained 
access to the migrant office by offering to help with the required annual 
evaluation report to EO. 

The TAGs have very little contact with the neglected or delinquent 
program, mainly due to tact that SEAs generally delegate their 
administrative respomvlbilities under the program to other state agencies 
(often the state department of corrections). The evaluation services 
mentioned in interviews were the prevision of workshops at annual 
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conferences on neglected or delinquent education and the development of an 
evaluation guide. 

Development of new materials and products In evaluation > TAG staff 
have little opportunity to create new evaluation materials. The major 
exception Is the TACs' development and adaptation of software for managing 
Chapter 1 data bases. In addition, TACs develop or adapt some materials on 
topics such as calculating sustained effects. TACs also maintain and 
disseminate the results of test equating studies and annotated descriptions 
of available tests. 

As TACs have reduced their provision of evaluation assistance, they 
have Increased their efforts to assist SEAs and LEAs Improve their Chapter 1 
services . 

Assistance In Program Improvement 

The scope of possible TAC assistance In program Improvement Is far 
broader than In evaltiatlon because the former Involves the heart of the 
Chapter 1 program- -the actual services provided to students. Moreover, 
within a single district, many different Chapter 1 programs may operate, 
creating many kinds of Improvement needs. (A single district may have, for 
example, Chapter 1 reading and math programs for grades K-3, a math program 
for grades 4-6, and a reading program for junior high students, with each 
program operating differently In each participating school.) In addition, 
program Improvement encompasses many diverse processes- -from Interpreting 
data to planning change, rethinking currlculiam and Instruction, Involving 
parents, coordinating with and Influencing the regular program, developing 
strategies for self -assessment at the teacher and program levels, and more. 

Unlike evaltiatlon. Instructional Improvement Is a topic that commands 
little professional agreement. Although educators generally agree on some 
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of the factors characterizing effective programs (e.g., strong instructional 
leadership, high expectations for students), there is far less agreement on 
how to transform an ineffective program into an effective one. 

In spite of these complications, both TAG and SEA staff report growing 
demand for assistance in J.mproving Chapter 1 programs. In our telephone 
interviews, SEA respondents described many more needs for assistance in 
program improvement than in evaluation. TAG staff said that the 
availability of program improvement help has created many new types of 
involvement with LEAs. As a result, TAGs face the following challenges in 
our view: 

• More demand exists for program improvement assistance than TAGs 
can meet, even taking into account those states that exhibit 
little interest in improvement. 

• Successful assistance , improving Chapter 1 services requires 
more than communicating ie principles of effective programs 
(which most SEA aiid LEA jaff already know) . It requires 
face-to-face, sustained help, either through a series of workshops 
or on-site consultations, activities that are time-consuming and 
expensive. Providing a single workshop or mailing out materials 
is not enouglh. 

• Program improvement assistance must also accommodate sensitivities 
and traditions stronger than those characterizing local evaluation 
practice. 

• As a result TAGs must balance intensity and duration of assistance 
against the numbers of LEAs and LEA staff whom they will help. 

TAG staff differ somewhat in their views on the kinds of assistance 

they should provide in program improvement. Most agree that generating good 

evaluation data is a first step; they also agree that multiple sources of 

data are needed to guide improvement efforts (for example, information about 

the regular program as well as Chapter 1 services and the degree of 

coordination between the two). Differences arise over how to go beyond data 

interpretation to actual guidance in improving Chapter 1 programs. Should 

TAGs recommend particular curricular or instructional approaches? Where 
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should they draw the line between recommending general strategies that are 
strongly supported by research and specific Instructional practices or 
materials? 

Just as TACs do not recommend particular tests, their contracts also 
prevent them from recommending particular curricula, materials, or 
pedagogical techniques. InsteaJ, they are expected to focus on general 
strategies, often by helping state and local staff review their program 
data, draw conclusions about program components that are weak (or strong), 
and develop plans to strengthen (or extend) them. In conducting these 
activities, TAG staff try to help LEAs recognize that they are responsible 
for making their programs as effective as possible , not simply for meeting 
all the legal requirements. 

Formats for program improvement a. sistance . The TACs have chosen to 
provide most of their improvement assistance in one or more of three ways-- 
consultations within a specific LEA, workshop series, and summer institutes. 

• On-site consultations have the advantage of focusing on a 
particular program in its own setting, with enough personal 
contact to support follow-up activities by phone or in person. 
The time required, however, limits the number of I-EAs and programs 
that can be assisted. 

• Three of the four TACs provide much of their assistance through 
series of workshops. Staff in these TACs said thaL a series of 
three workshops constitutes the best trade-off between one -shot 
workshops, which have little lasting effect, and on-site 
consultations, which can only be provided for a limited number of 
LEAs. 

• One TAC offers week-long summer institutes, which representatives 
of several LEAs attend. Here Chapter 1 staff participate in the 
equivalent of a five-day workshop, and TAC staff are available to 
answer specific questions throughout the week. The institutes are 
held on college campuses, and LEA staff receive academic credit 
for participating. 

Because workshop series are the most prevalent method of improvement 

help, they warrant special attention. Typically, a series of three program 



Improvement workshops is offered to teams from several districts. The 
number of LEAs, composition of the teams, location of the workshop, role of 
the SEA in the assistance, and procedures for selecting LEAs vary across 
TACs and states. The number of LEAs can range from one to over a dozen. 
The teams may include the local Chapter 1 coordinator, a Chapter 1 teacher 
and perhaps an aide, a regular teacher, and a Chapter 1 school principal. 
THc otA frequc^ntly suggests ^ere the workshops should be held; for example, 
an SEA may ask a xow performing district to host the workshops, knowing that 
otherwise they would not attend, or the SEA may have political reasons for 
wanting the workshops to be held in a certain region of the state. Workshop 
series and summer institutes are especially popular and efficient in regions 
with many small LEAs. 

Except for occasional subtle coercion by an SEA, LEA participation in 
the workshops is generally volxxntary. TAG staff report that it is difficult 
enough working with those who are interested in improvement, given the time 
constraints. Attempting to provide improvement assistance to unwilling 
recipients would likely be a waste of time, in their view. Those LEAs that 
volunteer must agree to send the required team and attend all three 
workshops in the series. 

Across the TACs, the workshop series tend to be structured in roughly 
similar ways. The first workshop typically focuses on self- assessment. 
LEAs bring their evaluation data and review and interpret it with help from 
TAG staff. In one region, for example, TAG staff present a graph showing 
state and national Chapter 1 achievement test means with space for LEAs to 
enter their own scores; this process indicates to LEA representatives that 
apparent annual gains in overall Chapter 1 achievement are not always 
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sufficient to keep Chapter 1 students from losing ground relative to other 
students. It also shows where the LEA stands relative to other districts. 

The second workshop, generally held a month or so later, leads the 
teams through (1) an analysis of their program's strengths and weaknesses 
using data elements and (2) the first steps in creating an improvement plan. 
At. LEA'S analyses may indicate, for example, that studen; 3 receiving Chapter 
1 reading services improved their achievement at a steady rate but that the 
rate of improvement dropped significantly when the services stopped* The 
rough outline of an improvement plan for this LEA coula include (1) greater 
Chapter 1 coordination with the regular reading and language arts program 
and (2) periodic follow-up services to former Chapter 1 reading 
participants • 

Because additional LEA staff usually heed to be involved in developing 
a plan, the team completes the plan between the second and third workshops, 
which are typically a month to six weeks apart* During the third workshop, 
LEA and TAC staff review the improvement plans jnd discuss potential 
problems in implementation* 

TAC staff itsually follow up with LEA participants after each workshop* 
This may include sending each LEA a reminder letter about the next workshop, 
providing written comments on improvement goals and plaMs, telephoning LEA 
teams to inquire about implementation problems and needs for further 
assistance, and inviting LEAs to participate in the series again* Several 
LEAs we contacted said that their SEAs had also followed up with them after 
TAC workshops* 

TAC staff report that in some states SEA personnel attend every TAC 
activity, including the workshops* Sometimes they attend simply to check on 
the services the TAC is providing; occasionally SEA staff also want to learn 
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how to conduct the workshops themselves using the TAC's materials. Although 
TACs cannot determine the quality and content of the workshops when someone 
else conducts them, they endorse and support the development of SEA 
capacities to help address the growing demand for improvement assistance. 
Relatively few SEAs have been willing to work with TACs in this way, 
however, according to TAG staff. 

Another way TACs sometimes increase their leverage is to act as brokers 
in providing improvement assistance. Tied into larger networks of education 
experts, TAG staff report that they sometimes match state and local needs 
with experts housed in SEAs, LEAs, diversities, federally sponsored labs 
and centers, and other institutions. 

Educational content on which assistance is based . In workshop series 
and in single workshops and consultations, TAG staff often present findings 
from recent research, according to our interviews with SEAs and LEAs. For 
example, TAG staff may summarize the research on students' time on task and 
teach Ghapter 1 teachers how to keep track of the amount of off- task time in 
their classrooms as a basis for increasing on- task time. Other research- 
based topics addressed In program improvement assistance include parent 
involvement, coordination with the regular instructional program, reading 
comprehension, and thinking skills. Several of the local Chapter 1 
personnel whom we interviewed placed particular importance on the TAG as a 
source of research information on effective compensatory education; one 
respondent said she speaks to TAG staff frequently to discuss **area£ of 
interest . . . and exchange Journal articles.** 

All four of the TAGs make use of the 13 principles of effective 
Chapter 1 programs developed by ED (and known as **the 13 attributes'*), 
according to TAC staff. This list includes seven organizational factors 

18 



(e.g., strong leadership, clear goals, parent involvement) and six 
instructional process factors (e.g., maximum use of academic learning time, 
high expectations, close monitoring of progress). The Chapter 1 director in 
one LEA said that a TAG workshop on the 13 attributes "brought theory into 
practice** and ** fired up** the workshop participants to assess and improve the 
district's Chapter 1 project. 

Other educational content used in technical assistance depends on the 
knowledge of TAG staff members. For example, in TAGs that employ a reading 
expert, improvements in reading comprehension may be a focus of assistance. 
Several SEA Ghapter 1 coordinators cited the Gommission on Reading's report, 
Becoming a Nation of Readers , as a guide that their TAG had shared. 

Although TAGs develop few major products due to a lack of time and 
other resources , they create materials used in the workshops as well as 
other materials used to generate interest in improvement. For example, one 
TAG has used the applications of successful projects under the Secretary's 
Recognition Program to develop a series of reports organized around the 13 
attributes. Each report in the series, entitled "In Their Own Words,** 
consists of application excerpts that illustrate the ways that projects have 
implemented ED's improvement principles. In addition to these reports, SEA 
respondents in our telephone interviews cited TAG workshop materials (e.g., 
handouts and overhaads) and research summaries as having been useful to 
them. One very large .LEA with whom we spoke adopted two workshops that the 
TAG developed on time management and student study skills and now conducts 
them for LEA personnal. 

C oordination of improvement assistance with state programs . States 
with their own improvement activities exert special pressures on TAGs. 
While the improvement -oriented states often have their own capacity for 
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technical assistance > they also often have a lengthy Improvement agenda » for 
which they may want TAG help* At the other extreme > TAG staff report that 
It Is difficult to deliver effective program Improvement assistance In the 
few states that have no Improvement agenda at all (usually states where 
Ghapter 1 officials define responsibilities In terms of enforcing 
compliance) • 

issues of coordination between TAG services and state Improvement 
efforts rarely arise » however » even In those states with heavily funded 
state reform programs » according to TAG staff. State Improvement efforts 
xisually do not reach Into the Ghapter 1 program > sometimes because of turf 
Issues at the SEA level and concern over Ghapter 1 compliance at the LEA 
level* (In addition^ state Improvement Initiatives often focus on changes 
xmrelated to Ghapter 1» such as Increasing teacher salaries and reducing 
class size.) Moreover » the level of TAG services Is too low to pose 
confllctrt with broader statewide efforts* In fact, because TAG staff work 
almost exclusively with Ghapter 1 staff, there are limited opportunities for 
overlap with state Improvement programs* One exception we found was ah SEA 
where the Ghapter 1 staff asked TAG staff to describe their Improvement 
workshop series to staff of the state Improvement program; the purpose of 
this activity was to share Information, however, not to coordinate services* 
Implementation of ED Improvement Initiatives * According to TAG staff, 
the Secretary's Recognition Program takes little staff time now and Is not 
viewed as program Improvement assistance by most TAG staff members* During 
the first two years of the Recognition Program, TAG staff helped LEAs with 
their applications, often at the SEA's request* This help focused on 
organizing and Interpreting evaluation data for Inclusion In the application 
r.-^d writing the description of the LEA project* TAGs still provide this 
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assistance occasionally but at a much lower level of effort than before; at 
most, some TACs conduct Information workshops for districts that are 
preparing applications and occasionally review program applications before 
submission to ED. 

Despite encouragement from ED, TAC staff tend not to focus their 
assistance on low performing districts. The primary reason Is that states 
are generally unwilling to Identify districts that are performing poorly. 
Sometimes their lack of willingness stems from a distrust of their own data: 
when one SEA used TIERS data to Identify the five best and five worst 
projects In the state, closer analyses revealed that the data on all ten 
were flawed and that the projects were performing at average levels. 
Another state told us that Identifying ineffective projects would be a waste 
of time because the real problem in these districts is a lack of money to 
hire qtiallfled teachers. In addition, as noted before, TACs are reluctant 
to work with districts that have not requested assistance. 

The exceptions to this trend are noteworthy, however. One state 
Chapter 1 coordinator described the TACs help in the state's three- week 
summer institutes, which are provided for Chapter 1 programs with low 
performance gains. Another state plans to require administrators and 
teachers from districts scoring in the lowest five percent on the state 
assessment to attend TAC regional workshops on Improvement, according to the 
state Chapter 1 coordinator. In addition, we contacted two LEAs whose SEA 
had quietly recommended that the TAC offer them assistance; both LEAs said 
that TAC help had b^en valuable. 

Assistance to Chapter 1 state -operated migrant education and neglected 
or delinquent programs . As in evaluation, the TACs provide very little 
assistance in program improvement to neglected or delinquent programs. 
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Migrant programs receive somewhat more help, but it is generally not 



provided in the same intensive formats (i.e., workshop series or on-site 
consultations) that are used with LEAs operating projects under the 
Chapter 1 basic grant program. Instead, program improvement assistance in 
migrant education generally consists of single workshops at annual meetings. 
According to one TAG director, typical workshops deal with **new ideas in 
reading comprehension, tiiae on task, setting high expectations, and math 
instruction." 

« 

Examples of how TACs assist LEAs In program Improvement , To obtain 
specific examples of how TACs can work with LEAs, we interviewed Chapter 1 
coordi''::ators and other administrators in five LEAs chosen from those where 
TACs said they had worked successfully on program improvement. The 
following descriptions summarize the services that TACs provided in these 
districts: 

• The TAC approached one large LEA at the urging of the SEA 
Chapter 1 coordinator, who was concerned about poor performance in 
several of the district's inner-city Chapter 1 schools. Working 
under the direction of the TAC, the LEA assembled sixteen school 
teams and central office staff members to participate in a series 
of TAC workshops, which addressed school-level self -assessment, 
the development of improvement plans, and analysis of plan 
implementation. The plans emphasized improvements in reading 
instruction, parent involvement, and student time on task. 
Between and after the workshops, the LISA maintained contact with 
the TAC and t^e SEA to discuss specific concerns and to 
communicate results. 

• Over a two-year period, 14 schools in a medium-sized city 
participated in two series of highly structured workshops that the 
TAC conducted. In the first workshop, ' **leaderihip teami;** assessed 
their schools' performance by analyzing their TIERS scores and 
their implementation of ED's 13 characteristics o£ effective 
Chapter 1 projects. On the basis of this information, the teams 
selected their "target areas" for Chapter 1 improvement. In the 
next two workshops the teams developed impro>^ement plans and 
evaltiated their implementation of the plans. The TAC stayed in 
touch with the schools between workshops and afterwards to help 
with implementation problems and answer questions. 
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• Another SEA asked the TAC to contact Chapter 1 project 
adainistrators in a small city> and the LEA agreed to assemble 
school teams to participate in a workshop series similar to that 
described in the first example. Two special concerns addressed in 
the workshops were the teaching of critical thinking skills and 
the early prevention of school failure. In addition, the 
workshops included discussions of improvements in achievement 
testing and other evaluation techniques » coordination with the 
district's Chapter 1 parent advisory council, and public relations 
within the community^ 

• The LEA of a medium-sized city participated in TAC workshops over 
a three-year period. Workshops in the first year focused on 

(1) analysis of improvement needs using local evalxxation data e^d 

(2) development of an improvement plan. In the second and th^^rd 
years, the TAC provided workshops to assist in implementing the 
plan, this assistance included help in improving Chapter 1 
reading instruction, selecting and txsing computer software for 
analyzing program data, designing a pilot program for Chapter 1 
parent involvement, and implementing the state improvement 
program. The LEA's evaliuition procedures permitted analysis of 
improvements in student time on task, matching those changes 
against improvements in Chapter 1 student achievement. 

• In a small LEA, the Chapter 1 coordinator contacted the TAC for 
program improvement assistance after attending a state-level 
workshop. The coordinator was impressed with the TAC's approach 
and invited the TAC to send someone to visit the LEA and discuss 
ways of improving the district's low Chapter 1 achievement rates. 
TAC staff met with the superintendent and principals, visited 
Chapter 1 classrooms, and then conducted several inservice 
training sessions for Chapter 1 teachers and other staff. The 
sessions focused on improving. student time on task through new 
curricular approaches, especially in reading. The approaches 
included (1) better use of data on individual student performance, 
(2) greater coordination between classroom teachers and Chapter 1 
staff, (3) more systematic selection of students to participate in 
Chapter 1 services, and (4) reduced use of pullout instruction. 

Changes in program improvement assistance . Although we looked for 

changes over time in the kinds of program improvement assistance that have 

been requested and provided, we conclude that it is too soon to see major 

shifts. We did discern that TACs are spending less time on the Recognition 

Program and more time on program improvement workshops. Host important, 

perhaps, TAC staff see a rapid increase in local requests for program 

improvement assistance. In many states, the improvement workshops have been 

the first contact between local program staff and TACs. LEAs' awareness of 
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the TACs' services in this area appears to create demands for nore 
assistance. This trend is reinforced by the districts that participate in 
the iaprovement workshops for the first time each year, thereby adding to 
the number of districts that ask for continued assistance. 

Allncatlon of TkC Services 

As the demand for TAG assistance increases, decisions on allocating 
services become more important. We fotind that TACs make three kinds of 
allocation decisions: how much service to provide each state, how to divide 
a state's services between the SEA and its LEAs, and what types of services 
to provide. TAG staff make these decisions in consultation with their SEA 
counterparts as part of the anntial negotiation of plans for TAG services. 
'Aiese plans are articulated in "letters of agreement" with each SEA. 

Allocation of Services Among States 

TACs' decisions on how to assign their technical assistance resources 
across statos takes several factors into account. The most important is the 
relatively low level of resources available under the TAG program. Data 
analyzed in this si:udy indicated that in the year ended September 30, 1987, 
the TAGS were staffed with a total of 42.3 full-time equivalent (PTE) 
personnel, of whom professional staff coni^tituted '^0.1 FTE, support staff 
made up 1.1 FTE, and clerical staff amoxmted to 11.2 FTE. These staff 
resources are spread among 50 states, the District of Columbia, the Bureau 
of Indian Affairs schools, Puerto Rico, and the Virgin Islands. The need to 
provide some level of service to each of these states and other entities 
imposes a low maximum amoxmt of service that can be provided to at\y one of 
them. Second, each entity must receive certain minimum amoimts of attention 
and service, including negotiation of a letter of agreement and periodic 
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couunlcatlon with the SEK^s link to the TAG (usually the state Chapter 1 
coordinator or the head of the SEA's evaluation office). The need to 
maintain this level cf comunication establishes a certain mininum level of 
service to each SEA* 

Within these upper and lower bounds » the TAG determines an overall 
level of service to a state based on its Chapter 1 enrollment, the state's 
past demand for TAG services » and any unusual circumstances such as a major 
state initiative relevant to Chapter 1. Because the service floor and 
ceiling moderate the difference in service level among states, more populous 
states tisually receive somewhat more service than other states but not in 
amounts proportional to their enrollment* From year to year, the overall 
amount of TAG service to each state remains about the same, unless there is 
a change in TAG funding level or state circumstances. 

The letters of agreement describe the aggregate amount of service that 
is planned for each state in terms of the number of days of service the 
state will receive. One TAG described the letter of agreement as the 
starting point for planning services; the director of that TAG said that 
more days can sometimes be found for a state, depending on the state's level 
of need. Another TAG director said that, while the letters of agreement are 
a starting point, his primary method of determining allocations of TAG 
effort is '*first come, first served.** He meant that the TAG is mainly 
interested in responding to state needs and that he will find a way to 
address the most pressing needs in ea^h state. TAGs often reallocate 
resources within a state during the year, either from one t3rpe of SEA 
service to another or from the SEA to LEAs* 
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Allocation of Stervlcgg Between an SEA and the Staters LEAs 
Decisions about allocating services between the SEA and the state's 
LEa\s are also negotiated in the letters of agreement, although the level of 
detail varies. The aajor factor detemining these decisions is the 
preference of the SEA, according to TAG staff. Sone SEAs prefer to receive 
the bulk of assistance thenselves, while others encourage the TAG to deliver 
services directly to LEAs. According to one TAG director, the reduction of 
TAG sf^rvices in the early 1980s proapted many SEAs to eliminate TAG services 
to LEAs, and in many cases the SEA has not reversed these decisions. 

Among tile SEAs that allow the TAG to assist LEAs, many want to be kept 
informed and, in some cases, offered the opportunity to participace in the 
service* Other SEAs ask that the TAG work directly with LEAs, without 
involving tht SEA* In the latter group of states, the TAG may be asked to 
work with LEAs on a "first come, first served** basis xrntll the state's 
allocation is exhausted, or the SEA may negotiate a specific number of days 
for certain LEAs* 

SEAs that want the TAG to assist LEAs directly may or may not specify 
which LEAs are to be assisted. Some states target small districts to 
receive services, usually because they are less likely to have relevant 
expertise or access to other resources. Oth'^rs target only very large 
districts ) because of the nuiiber of Chapter 1 students they serve* -and 
perhaps also because they wield political power in the state. As noted 
earlier, a few states concentrate a portion of TAG se^^ices on ineffective 
districts* 

Decisions on What Types of Servlcos To Provide In a State 
SEAs largely determine the services they receive from the TAG, due to 
the TAGs' orientation to ;ards the SEA as their main client. However, 
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factors other than their own needs influence SEAs' requests for service. 
These include their perception of the capabilities of the TAG and their 
awareness of the constraints that ED imposes on the TACs. For example, 
several TAG directors said that SEAs requested more program improvement 
assistance once ED said that TACs should devote at least half of their 
resources to that area. 

TACs also report that they have pushed SEAs towards requesting help in 
program improvement. But, as one TAC director said, **Even when we are 
proactive, we react to the way the states want it handled.** 
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TAG Administration 

We turn now to look at the activities and capacities that combine to 
create the administrative arrangements for providing TAC services. These 
include the organizations in which the TACs are housed, the staff employed 
to deliver services, their management procedures, and their use of funds. 

Organizations Qperattny the TACs 

Seven organizations currently participate In operating TACs. One 
organization, Northwest Regional Educational Laboratory, operates both the 
headquarters and field offices of one region, but in the other three regions 
the headquarters and field offices are operated by different organizations. 
In these regions, a prime contractor operates the TAC headquarters office, 
and a subcontractor operates the TAC field office. Educational Testing 
Services (ETS) is the prime contractor in two TAC regions but operates the 
two contracts out of different ETS regional offices; 

In general, we found relatively little contact between the three prime 
contractors and their subcontractors. In each of the three TACs that have 
subcontractors, the two offices work independently, preparing separate 
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budgets and conducting management activities independently* Prime 
contractors and subcontractors communicate mainly on matters such as 
preparing reports to ED and monthly invoices. Although they may 
occasionally coordinate on technical matters, the amount of such 
coordination is about the same as that between TACsi which is relatively 
little. 

Surprisingly, the prime contractors do not necessarily provide greater 
amounts of staff effort than their subcontractors. Two field offices report 
slightly higher levels of effort (based on PTEs reported for the year ended 
September 30, 1987) tt'^n the corresponding headquarters offices. In these 
three regions, the headquarters offices reported PTEs ranging from 7.1 to 
4.4. Effort expended in the three field offices ranged from 5.5 to 4.6 
PTEs. (Across all eight TAG offices, effort levels ranged from 7.4 to 3.1 
PTE staff each, averaging 5.3 PTE staff per office.) 

Each of the seven organizations in which TAG offices are housed 
conducts applied research or related activities for other clients. Several 
of the organizations conduct extensive research and te>hnical assistance in 
areas directly related to the work of the TAG. The extent of those 
operations is relevant to TAG operations in several respects: 

• TAGs operating in organizations with large staffs engaged in 
educational research and technical assistance are able to assign 
qualified staff to the TAG as needed on short notice and for small 
amoxmts of time on specialized assignments. 

• The other work of these organizations (e.g., curriculum 
development, computerized data base development, testing) 
contributes to the technica! capacity of the TAG in some 
situations. 

• These organizations also offer (1) career ladders to TAG staff 
with ambitions to move into senior administrative positions in 
educational research and assistance and (2) professional 
colleagues with whom TAG staff can try out ideas and discuss 
service*related problems. 
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• On the other hand, smaller orgwlzatiotis or organizations with no 
other work in TAC-related fields offer greater staffing stability, 
since staff are less likely to be reassigned or promoted to other 
work. 

The professional advantages offered by the large, education-oriented 
organizations can be achieved in other ways by other organizations, however, 
such as through the development of professional networks complementing the 
capacities of the TAG staff. 

Characteristics of TAG Staff 

Our review of staffing looks at both staff assignments and the hiring 
of TAC staff. 

Staff assignments . TAG directors assign staff according to the needs 
and characteristics of the states the TAG serves. One or two staff members 
are usually assigned to work with each state, although they may each be 
spending far less than full time on the state. The staff member who is the 
lead contact with the state generally maintains regular communication with 
his/her SEA counterpart and arranges for all the TAG services needed in the 
state. Where appropriate, the TAG staff members responsible for working 
with the state actually deliver the services that the state uses, although 
sometimes they serve as brokers in arranging services that others with 
relevant qtialif ications provide. 

Most TAG staff members have lead responsibility for one or two states. 
The exception is staff members hired to work with all the states in a 
region. Data base developers, for example, are usually qualified to work 
only in that area and are expected to work with all states needing help in 
developing or maintaining a Ghapter 1 data base. 

TAG directors report that states value continuity in TAG staffing, so 
that they try. to maintain the same state assignments for as long as 
possible. They also try to match the TAG staff contact with the needs and 
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characteristics of the state. For example, some states may place a high 
value on systems expertise or, at the other end of the spectrum, innovati^^e 
instructional strategies; TAC directors try to respond to those interests by 
assigning staff who have the qualifications or characteristics that the SEA 
values. In general, TAC directors also try to assign more senior staff 
(often themselves) to the largest states. 

One factor limiting flexibility in staff assignments is that TACs 
conduct no ongoing formal training of their staff members. Newly hired 
staff are tjisually assigned to work with a more experienced staff member for 
.the.,first month» and then they are; expected to begin delivering services on 
their own. After the first month's experience, staff aire given no further 
formal training in technical assistance methods. Because of the demand for 
service delivery^ TAC staff generally have no time to devote to reading 
professional Journals in educational evaluation and program improvement. 

TAC staff are usually organized in L flat structure » with all 
professional staff reporting to the director of the TAC office. The 
director usually has lead or sole responsibility for reviewing each staff 
member's performance annually and determining salary increases. 

Considerations in hirin^^ TAC staff . TAC directors indicated clear 
preferences on the qualifications they look for in hiring new staff. These 
include the following: 

• Strong interpersonal skills, which should translate into the 
ability to xmderstand and talk to SEA and LEA staff, 

• Familiarity with Chapter 1 or other educational programs at state 
and local levels, 

• Evaltiation skills, 

• Familiarity with educational improvement methods, including 
current research in the area, and 

• Personal commitment to the goals of the TAC. 
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TAC directors said that they do not consider it essential for new TAC 
employee» to have strong educational measurement qualifications. So long as 
at least one person In the office has strong skills In this area, that 
Indlvldtial can serve as a resource to the entire office* 

They reported Increasing difficulty In recruiting qtiallfled staff 
members and retaining current staff, due to competition from LEAs. All of 
the TAC directors said that LEAs now pay higher salaries than they can pay 
on the TAC contracts, so that they must recruit much more extensively than 
before. Several TAC directors also reported problems of retaining staff 
members, due to "btxm-out," resulting from-pressures- to provide more 
services than the TACs are staffed to provide* 

Another consideration In hiring new staff Is that the ED project 
monitors review each proposed new hire before the TaC can make a job offer. 
Because that office's staffing priorities sometimes differ from the TAC 
director's (e.g., more ED interest in academic credentials), a TAC must 
sometimes reject a person whom the director finds qtialifled because the ED 
monitors might not find him/her acceptable. 

Management Procedures That TACs Use 

The main management technique that TACs employ is tracking the use of 
professional time. They use several tools in this process, as follows: 

• Some of the TACs track the use of days allocated in the letters of 
agreement. One TAC office sends quarterly reports to each state, 
enumerating the days spent and the days available for the next 
quarter . 

• All TACs require each staff member to maintain time sheets, which 
use the TAC contract tasks to describe how the staff member's TAC 
time is spent. The time allocations are summarized; at least 
monthly to determine how the staff as a whole 1^^ using its time. 
This Information is required to be submitted to ED. 

• TAC directors monitor costs by reviewing monthly expenditures and 
approving in advance all expenditures for travel and non- labor 
direct costs over a certain minimum. 
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Using this information, each TAG director determines whether too much 
or too little effort is going into the TAC's various activities. As needed, 
the director reassigns effort among states, staff, and activities. Our 
review of aggregate time allocations for each of the TAG offices indicated, 
for the year ended September 30, 1987, how TAGs divided their staff time 
across the activity categories in the current TAG contracts. Table 1 
presents the percentage of staff time allotted to each category, along with 
the highest and lowest allotments among the four TAGs. 

As the table shows, TAGs spend almost three-fourths of their staff time 
deUverihg. technical, assistance, with over half of .that assistance, aimed at 
program improvement* Althoug^h percentages for evaluation and program 
improvement assistance vary across TAGs, our interviews with TAG staff 
indicate that much of that variation reflects differences in how activities 
are classified* 

The other major TAG activity is project management* We found that 
some TAG directors classify most of their supervisory time as management, 
while other directors consider most supervision to be part of the TaG's 
technical assistance services* 

Use of TAG Funds 

Analysis of TAG hourly costs indicates that one hour of TAG time in the 
year ended September 30, 1987, cost an average of $35.38, including 
applicable indirect costs and excluding direct costs such as travel, 
materials, and incidentals. The highest average hourly cost among the four 
TAGs was $40.05, and the loweist average cost was $31.76. Broken down by 
personnel level, the average hourly cost for professional time across all 
four TAGs was $41.57, with $25.26 an hour required for the time of support 
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Table 1 



Time Spent by the Four TACs on Each Major TAG Activity 

(in percents) 



Activity TAG Average Highest Percent Lowest Percent 

Letters of agreement 11 0 

Technical assistance in: 

evaliiation 33 40 28 

Technical assistance in 

program improvement 40 46 35 

Technical investigations 12 0 

Additional tasks ;(a)" 6 9 2 

Maintenance of staff 

expertise 4 7 1 

Outreach and awareness 3 5 1 

Management 10 18 4 

TAG directors^ meetings 2 2 1 

Total 100 

(a) Includes clearinghouse for workshop materials, test information center, 
special jpresentations or workshops, and development of materials to be 
used across TAGs. 



persc.inel and $19.67 an hour for clerical time. Gompared to hourly costs of 
firms and nonprofit organizations that provide comparable educational 
assistance and research services, these costs are reasonable. 

We analyzed the allocation of expenses between labor and other direct 
costs and f otind that the TAGs used an average of 80 percent of their 
resources for labor- related expenses and 20 percent for expenses related to 
other direct costs. Only the TAG located in the region requiring the most 
travel used more than 20 percent of its funds for other direct costs. 
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Conclusion 

As this chapter documents, the TACs provide a broad range of services 
Intended to help SEAs and LEAs Improve the quality and effectiveness of the 
Instruction provided In Chapter 1 projects. TAC services address these 



the Information available for assessing educational effectiveness. Despite 
varliatlons across TACs, they seem in general to have selecteci technical 
assistance strategies that are consistent with their program goals. 
.Mqrtover,, these- are implemented within administrative frameworks that 
generally support the delivery of the intended services. 

The major exceptions to this picture of a coherent, efficient system of 
technical assistance are: 

• TACs continue to provide some services \e.g., explaining TIERS, 
helping prepare SEA reports) that SEAs should be able to handle on 
their own. 

• TAC assistance in program improvement does not consistently 
Involve SEAs in a way that helps them improve their own 
capabilities. 

• The availability of TAC assistance in program improvement is 
creating demands foi services that may exceed the current system's 
capacity to fulfill. 

• Large states are not receiving an equitable share of TAC service, 
due to the high floor of service necessary for each state and the 
low aggregate level of service. 

• TAC offices have relatively little contact with each other and 
thus have little opportunity to share materials and approaches. 

• TAC staff receive little or no formal training in the provision of 
technical assistance. 

To understand more about the context in which TAC services are 

delivered t we turn now review the characteristics of SEAs and LEAs that 

influence their needs for and use of the technical assistance TACs provide. 



improvement goals from two angles, directly through technical assistance in 



improving programs and indirectly through technical assistance in improving 
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3. The Users of TAC Assistance 



TAC services are Intended to extend and Improve the capacities of SEAs 
arid school districts » especially in the Chapter 1 and evaluation offices. 
Because users call on the TACs to help thea solve specific problems or to 
teach them new skills, the TACs' work reflects the expertise, preferences, 
and habits of these agencies. Recent research, including other Chapter 1 
studies as well as this one, has described the staffing, capacities, and 
activities of state and local Chapter 1 offices. We draw on our research 
:;and the recent -literature, to discuss who uses- (or could- use) the TACs , what 
they are doing in the areas of program evaluation and improvement, how they 
use technical assistance, and bcnh the actual and potent;.al contributions of 
TACs to their work. 

Users at the State Level 

This discussion examines SEAs' capacity and activities in the 
administration of/Chapter 1 and how TACs can supplement their work. 

Staff Capacity 

Some 600 FTE st^ff members work in state Chapter 1 offices, according 
to a survey conducted in 1986 for the National Assessment of Chapter 1 (U.S. 
Department of Education, 1987). This number, based on a survey of 49 SEAs, 
represents a decline from the last year of Title I, as shown in Table 2. 
The largest category of SEA employees working on Chapter 1 is that of 
**generalists,** numbering 3J0 in the 49 SEAs, as the table shows, or an 
average of six to seven PTEs per state. These staff members include the 
state Chapter 1 director and others who review local Chapter 1 applications, 
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Table 2 

Number of SEA Staff Under Title 1 and Chapter 1 



Number of PTEs Percent Change 

Title 1 Chapter 1 (1981-82 to 

Functions (a) ^1981-82^ (1985-86^ 1985-86^ 

Generalist (b) 466 330 -29 



Specialist (c) 157 113 -28 

Subject specialist 32 26 -19 

Parent specialist 10 3 -70 

Evaltiation specialist 32 27 -16 

Audit/fiscal specialist 83 57 -31 



Other (d) 48 21 -54 

Secretarial 212 141 -33 

Total 881 605 -31 

(a) Data were collected from 49 SEAs. 

(b) These are staff who have general oversight responsibilities for 
Chapter 1 operations in particular school districts. This number 
includes the state Chapter 1 director. 

(c) The ntunber of states reporting specialist functions varies. 

(d) Examples include information writer » officer manager, administrative 
assistant; and attorney. 



Source: Stat:u survey conducted for the Chapter 1 National Assessment, 
i985-86. 



monitor local programs, answer que&,tions about regulations, and coordinate 
technical assistance from the state. Chapi 1 offices also have an average 
of just over two FIE "specialists," about half of whom :?pecialize in fiscal 
matters. Thu remaining specialists include the people who assist districts 
with evaluation, curricultim and instruction, and parent: involvement. 

Chapter 1 evalaation is the specialty of about 27 PTE staff in state 
Chapter 1 offices. Some offices have a full-time professional working on 
evaluation; more have a staff member who divides his or her time between 
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evalxxatlon and other responsibilities » such as monitoring districts; in some 
states no one in the Chapter 1 office specializes in evaluation, and experts 
from other offices handle this ftinction. 

Staffing levels are similar for Chapter 1 subject specialists, who 
number about 26 FTE nationwide. Some states have at least one staff member 
whose job is to work in an area such as Chapter 1 reading programs. Other 
states' Chapter 1 offices rely on the subject-matter experts in other 
bureaus of the SEA* 

Although there is no ''typical'' SEA, the findings of a set of case 
studies of state and local Chapter 1 administration provide a generalized 
description of the staff characteristics from a sample of 20 SEA Chapter 1 
offices (Farrar & Millsap, 1986). Hany staff members have lengthy tenure in 
die SEA, and most were trained within the Title I program framework. Often 
the longer* tenured staff are especially proficient in regulatory issues, 
while "more recently hired SEA staff tend to have expertise in such areas as 
reading, curriculum development, school impro\ement or effectiveness, and 
evalxxatlon" (Farrar & Millsap, 1986, p. 23). SEA staff members commonly 
have experience at the local level, in positions t? at include teacher, 
principal, or curriculum specialist, as well as local Chapter 1 director. 

SEA Staff Activities 

Host SEA Chapter 1 staff members concentrate their efforts on ensuring 
local compliance with the law and regulations (U.S. Department of Education, 
1987; Farrar & Millsap, 1986). Monitoring programs on-site, reviewing local 
applications, and advising districts by telephone on the acceptability of 
particular practices are traditional state activities that absorb staff 
time. Even among the newer staff members with particular expertise in 
educational content, most are being trained as generalists to concentrate on 
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regulatory compliance. In addition, however, SEAs report that they spend 
time on the issues of evaluation and program improvement that TACs help 
with. 

Many SEA activities related to evaluation are simply the clarification 
and oversight of procedural requirements (Farrar & Millsap, 1986). The SEA 
Chapter 1 directors whom we interviewed have continued to use TIERS, 
endorsing its value to them and national audiences as a way of evaluating 
Chapter 1. The use of TIERS was required by 39 out of 49 SEAs at the time 
the National Assessment surveyed them in 1986. In addition, some SEAs 
analyze raw scores for their districts, and some compile reports that may 
include an analysis of effective program approaches. Some offer workshops 
or other assistance to encourage evaltiation use and connect evaluation with 
program improvement. Some of the SEAs we interviewed use the TIERS data to 
inform their monitoring. The Chapter 1 director in one state said that, if 
local scores indicate little or no gain, monitors will work with the local 
staff to "analyze** the program. 

In large states or those with a special interest in evaluation, the 
staff that works on these activities may be sizable and technically 
sophisticated. However, with a nationwide average of less than one PTE 
employee per state overseeing evaluation requirements as veil as offering 
assistance, the scope of assistance available from most states is 
necessarily limited. Furthermore, a TAC staff member told us. '^A lot of 
state evaluators don't have much formal training in evaluation. Many 
inherit the jot by default. Others know evaluation but don't know the 
Chapter 1 program.** This staff member said that across five states. Just 
two state evalxiators have ••any testing expertise. •• 
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The SEAs in our sample reported a variety of efforts to improve 
Chapter 1 in their states. They typically offer workshops for Chapter 1 
administrators and teachers, which principals and other staff may also 
attend. Others mentioned their participation in the Secretary's Recognition 
Program. However, the likely effects of this participation on program 
improvement seem limited. Submitting applications, which many states dOt 
does not benefit any districts other than the one applying. Tvo states 
mentioned that they disseminate information on model programs, but none 
mentioned dissemination or implementation efforts that went beyond the 
transmission of information*-a notoriously weak lever on program 
improvement. 

SEk staff reported that they are working on several issues in Chapter 1 
improvement, including coordination with the regular program, parent 
involvement, use of computerized data bases, and various state-designed 
procedures for reviewing and using data on program effects. Some said that 
they tise evaluation data to identify low scoring districts and provide extra 
assistance to these districts. Some have used the research on effective 
schools, including the list of 13 attributes, as the basis for a recommended 
process of local self* assessment. 

The amount of time an SEA devotes to program improvement varies, but 

the findings of Farrar and Millsap indicate that it is often limited. Those 

researchers identified two types of SEAs, which they called ^traditional** 

and "changing." The traditional states, emphasizing compliance and often 

adhering to the Title I requirements that Chapter 1 removed, were not 

oblivious to program quality but in practice did little to further it: 

In conversations with SEA Chapter 1 staff in traditional states about 
the technical assistance they provide to improve programs, most spoke 
of their efforts to clarify the law and to help districts modify 
program designs, materials, and documentation to achieve compliance. 
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Rarely did they Mntion technical assiscnnce in terns of curricular or 
instructional areas, Sooie staff- •particularly never aembers with 
backgrounds in curriculua and instruction- -offered program improvenent 
suggestions to teachers, and ran ad hoc technical assistance sessions 
, during aonitoring visits. But instances of this kind of substantive 
help vere rare in traditional states. Annual statewide. Chapter 1 
Beatings usually incliide sessions devoted to instructional issues and 
techniquas that work with youngsters. But insofar as technical 
assistance for program ioprovaMnt is thought to require qualified 
assisters and sustained vork, traditional state Chapter 1 offices 
provide virtually no prograa related technical assistance, (p, 131) 

The^ ^changing" states in that study did isore to teach districts how to 
analyse and inprove their prograns, to disseminate infomacion on effective 
prograas, or to create requirements or incentives for local improvement, 
Becatxse fJhese efforts vere nev, their local effects might not yet have had 
time to appear, Hovever, at t!ie time of the research, district staff 
characterized state help vith their programs as modest at best. Of 27 
districts in nine states (six " traditional and three '^changing'*) , 8<h 
districts reported that the state had helped vith program 7*iality, 

Fairar and Millsap found that SEAs have adapted their activities in 
different vays over the past fev years, as their staffing levels have 
decreased. Some have made a policy decision to de-emphasize program 
mechanics, and some of these states nov report that they use more of their 
xfesoturces to vork vith districts on program development and quality. Others 
have attempted to maintain the same level of attention to the details of 
compliance, reducing their assistance to districts in areas such as 
ciurriculum and parent involvement. 

In simary, although staff meoibers in the SEAs ve contacted take a real 
interest in evaluation and program Improvement, other research strongly 
suggests that they spend relatively little tima on tV ^ issues and that 
districts do not find SEA Chapter 1 offices especially helpful on matters of 
program quality, Becatise state officials ar;e busy vith the routine 
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adalnlstratlon of the program^ they appreciate having TACs available to help 
with evaluation and program Improvement* 

How TACs Can Fill Gaps In State Capacity 

By devoting at least part of an FTE to providing help in each state » 
the TACs significantly Increase the total amount of expert staff time 
brought to bear on evaluation and Improvement. Unlike many of the content 
experts on SEA Chapter 1 staffs, they are not responsible for ensuring 
compliance and can devote their full attention to assistance. And In fact a 
considerable share of the help they provide to states consists of acting as 
extensions of state staff. Coordinators In some states expressed their need 
and appreciation for the TACs In just these terms, saying, **[We need] more 
warm bodies," and "We're short of staff." 

Althougjli the TACs do contain experts in evaluation and program 
improvement, the level of expertise required to meet the bulk of state and 
local needs does not Involve great technical sophistication. A state 
coordinator gave these examples of the technical issues the TAC helps 
address: "selecting test Instruments, interpreting data, using the state 
test for Chapter 1 purposes" --none of which would pose a real challenge to 
an evaluation expert. 

SEAs appreciate the TACs' contribution in developing the materials and 
procedures that SEAs use with their districts on issues of evaluation and 
Improvement. For example, one TAC helped develop a manual on multiple uses 
of achievement test data. Another helped a state develop its criteria for 
assessing exemplary Chapter 1 prof^ams, and still another participated in 
developing a self-assessment procedure that all districts in another state 
must use. The SEA coordinators expressed a wish for more developmental work 
of another kind from TACs--the Identification or creation of "models" of 



effective local Cliapter 1 programs. Four of the nine state coordinators 
whom we Interviewed said they want to know more about effective program 
models; two of these mentioned that they want models that are alternatives 
to pullout designs. 

Something the TACs do not do much, but would like to do, Is Inservlce 
training for SEA staff. In both evaluation and program Improvement^ 
Inservlce from the TACs could potentially bolster the knowledge and skills 
of SEA Chapter 1 staff. TAC staff members commented that they would like to 
conduct training on how to facilitate local program Improvement- -which this 
study, like that of Farrar and Hlllsap, confirms as an area of need in the 
SEA Chapter 1 offices; 

Users at the Local Level 

This discussion looks at the same Issues as the preceding section but 
focuses on school districts. 

Staff Capacity 

Because local Chapter 1 programs vary so much in size and complexity, 
their staffing varies as well (Farrar & Millsap, 1986). The majority of 
p art i'^ipa ting school districts have programs small enough to be managed oh a 
part-time basis by a coordinator who may be the superintendent, a principal, 
or a member of the central staff. Many other districts have one or two 
Chapter 1 FTEs in the district office, with various ways of dividing the 
responsibility for the regulatory and instructional aspects of the program. 
The largest districts have Chapter 1 programs with dozens of FTE staff 
members, including budget specialists, monitors, evaluators, and niamerous 
ittstructional supervisors. 
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Across the approximately 14,000 school districts with Chapter 1 
prpgrems, the National Assessment estimated that the program's 
administrators include 3>625 PTE coordinators, 1,422 FIE curriculum 
specialists, 363 FTE evaluation specialists, and 349 FTE parent specialists. 
(District Chapter 1 offices also contain fiscal specialists, clerical 
personnel, and others.) Thus, although some large districts can oversee 
Chapter 1 with a number of people who are knowledgeable about evaluation and 
program improvement, most districts have few staff hours to devote to these 
matters • 

District Activities 

Like SEAs, district offices devote much of their Chapter 1 staff time 
to implementing required procedures and to creating and maintaining records 
that demonstrate compliance with the law* The mechanics of selecting 
schools and students, developing ah application, preparing for monitoring 
visits, patrolling for violations of the supplement-not-supplant 
requirement, and filing reports all consume staff time. Also like SEAs, 
however, districts have staff members who try to attend to evaluation and 
program improvement. 

Local evaluations, according to the National Assessment, continue to 
follow TIERS; about 90 percent of districts follow one of the Title I 
evaluation models and send their results to the state as often as before. 
The local use of evaluation results is more variable. There are cases in 
which the availability of program information directs attention to a problem 
and leads to program change. More commonly, though, Knapp et al. found that 
Chapter 1 coordinators think evaliiation "has relatively little to do with 
the decisionmaking process** (1986). These researchers cite a typical 
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example of the local response to a state requirement concerning evaluation 
use: 

The state requires the district's application to include a narrative 
describing the way evaluation results have shaped this year's program. 
The Chapter 1 director writes that narrative but indicated that 
evaluation hasn't influenced any real decisions » because the **test 
results aren't that bad." (p. 46) 

Knapp et al. observed that some local circumstances make the tise of 
evaluation data more likely. These include the presence of local staff with 
evaluation expertise, large district size, central control over Cb 'jpter 1 
decisions, attention to testing (e.g., due to a state testing program), pre- 
existing debate over particular program decisions, and commitment to 
information-based decisionmaking among top district leaders. 

Although states vary in the extent to which they encourage evaluation 
use, Farrar and Millsap did n ^ find that greater state efforts were 
associated with greater district use of evaluation. Like Knapp et al. , they 
concluded that the determinants of evaluation use were primarily local. 

While some districts are oriented to changing their Chapter 1 programs, 
others are not. Usually, the program looks very much the same from one year 
to the next (Knapp et al., 1986). Its stability comes in part from the 
normal inertia and adherence to standard operating procedures that 
characterize ^ny organization, but also in part from the fact that ChaT>ter 1 
and Title I have a long history of regulations and monitoring. Once a local 
program meets with the approval of state Chapter 1 officials, local 
decisionmakers are sometimes reluctant to tamper with it. When a program 
does change, the factors that contribute to change may include evaluation 
results but may also reflect professional trends and state reform 
initiatives • 
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The Influence of state reform Initiatives is probably felt more 
strongly imong local Chapter 1 staff tnan sta.e Chapter 1 staff because 
these efforts, often spearheaded by a governor or chief state school 
officer, tend to bypass the federal program offices in an SEA. In the 
latest wave of school reform, new testing has led to the most noticeable 
local effects. Knapp et al. found instances of local decisionmakers 
significantly redirecting their Chapter 1 programs in an effort to improve 
scores oh a state -mandated test: they have chosen to serve different 
grades, add a new subject, or undertake decailed revisions of the regular 
curriculum (which often drives the^ Chapter 1 curriculum) so that students 
will be better prepared for the test. Other types of state reform 
initiatives have also had effects ^ Some states' emphasis on school -based 
improvement has led to specific suggestions for coordinating Chapter 1 and 
the regular curricultun, for example. 

How TACs Can Fill Gaps in Local Capacity 

Uiillke SEAs, school districts are simply too numerous to use TaCs as a 
significant extension of their own staff* Our analysis suggests that the 
greatest potential benefit of TAC help for district staff is (1) having 
their attention directed to issues in evaluation or program Improvement and 
(2) developing their own staff capacity to deal with these issues* 

At the most routine level, the TACs can help districts by answering 
specific questions through telephone consultations or workshop 
presentations. When we asked the local Chapter 1 coordinators in a 
stratified random sample about help they had received from the TACs, most 
gave examples of this type of help. They remembered that the TACs had 
answered questions or helped them with a routine, often required, function. 
For example, one local coordinator had needed help filling out the state 




form on sustained effects; after attending a TAG workshop, the coordinator 
knew how to fill out the form. Another received TAG help in filling out an 
application for the Secretary's Recognition Program but said that the 
district has its own inservice offerings and did not want other help with 
program improvement. 

TAG staff told us about the process of bringing more extensive » 
substantive help to districts. In many cases, they said, the triggering 
event has to do with the local testing prop*:am. A district asks for help 
after updating the test (**some LEAs are using, tests that are ten years old,** 
said a TAG staff member) or changing to spring- to- spring testing. In^the 
latter case, the reduction in apparent student gains brings request? for TAG 
help In interpreting the test data for the school board and parents. 

Once a district begins to explore issues in evaluation and program 
improvement, the TAG can sometimes become an effective advocate for 
Improvement simply by helping program staff see their options. A TAG staff 
member said: 

Ihe new local directors are not sure where their power lies. For 
example, they don't know that more than pulloul: is possible, which you 
learn when you discuss program setting. They don't know that they 
don't have to have six Ghapter 1 teachers at a certain school, even 
though it's always been dene that way. You have to realize that 
Ghapter 1 directors in smaller districts have little power relative to 
their principals. 

TAG staff report that they find a void in Ghapter 1 instructional 
leadership in a number of districts they work with. While district staff 
assume that principals are taking responsibility for the program, principals 
may consider Ghapter 1 a centrally directed program that they are not 
supposed to influence. As a result, the Ghaptei 1 teachers and aides are 
left on their own to carry out the program. 
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Using Techntcal Assistance 

When state and local Chapter 1 staff members want help, where do they 
turn? Our data and other analyses address these questions* 

Sources of Assistance 

For the SEA staff members we interviewed, the TACs are the chief source 
of technical assistance. Many SEA respondents also reported that they 
obtain help from regional laboratories and other offices within their own 
agencies. Other sources of assistance, mentioned by smaller numbers of 
respondents in this nonrepresentative sample , included local \miversities, 
state lobbyists in Washington, and Chapter 1 coordinators in other states. 

For districts Hi this sample, the SEA is the major source of help. Ttie 
assistance comes from the Chapter 1 office and other parts of the agency, 
such as the testing office. Several respondents mentioned receiving help 
directly ^rom a TAC. Our analysis of the interview data also indicates that 
some of the respondents who cited SEA help had benefited either directly or 
indirectly from TAC help, such as a workshop xmder state auspices with 
presentations by TAC staff members or SEA presentatiotis using TAC analyses. 
Other sources of help include other offices in the district. Chapter 1 
offices in other districts, a regional lab, and professional associations. 

The sources of help other than TACs, then, reflect the communication 
patterns commonly found within and between organizations: people talk with 
their colleagues in different offices of the same agency; they talk with 
their cotmterparts in other agencies; they keep up with professional 
associations; and they attend to vertical program networks (district staff 
ask SEA staff fo^ information) . Of the special-purpose assistance providers 
that the federal government supports, the TACs are the ones used most often 
within Chapter 1. 
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An SEA Cash Grant as an Alternative to TAG Assistance 
Because ED wanted to know how SEAs would react to the idea of receiving 
a cash grant as an alternative to receiving TAG assistance » we asked them. 
Of the SEAs we interviewed » nearly all responded to this question by 
reiterating their satisfaction with TAG services. Several said that a cash 
grant would be a poor idea; one of them said that it would disrupt the 
valuable continuity of services that the TAGs provide, while two pointed out 
that as federal contractors the TAGs play a useful role as intermediaries 
between the SEAs and ED. Another SEA respondent said they would use a cash 
grant to hire the TAG staff, adding, "but don't give us the money. Any 
discretionary money ends up in the superintendent's office. We'd never be 
able to hire who we want or spend it the way we might want to.** 

Among the SEA respondents who were willing to entertain the idea of a 
cash grant, most said they would look for service providers like the TAG 
staff. They (id not seeui to have specific ideas about where to find such 
people. Their views were summed up by the answer, **We'd go out and get 
somebody to offer the same sort of thing.** In one SEA, however, this 
question elicited a criticism and a clear alternative: **We could be much 
more selective in the people we would hire, you know, shop around for the 
right staff for tis, not just take what's offered.** Thiji SEA would continue 
to use some evaluation help from the TAG but would hire more SEA staff and 
use some xmiversity*based consultants. However, at least among the nine 
SEAs in our sample, this was an xmcommon point of view. 

Conclusion 

In assessing state and local needs for assistance in Chapter 1, the 
nuinbers alone t:ell a story. Staff levels are quite low, in both state and 
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local Chapter 1 offices » In relation to the challenges of evaluating aad 
Improving, services In this large program • Perhaps even more Important Is 
the program's regulatory tradition, which dictates that substantial amountr 
of staff time and attention must go to maintaining compliance with legal 
requirements. Evaluation and Improvement are not the central concemr? of 
most Chapter 1 staff- -although they command considerable respect as abstract 
notions. In this context, the expertise and availability of the TACs cannot 
help but make a welcome contribution. 

Looking at the match between what TACs offer and what SEAs and 
districts want, we see that TAC staff play three different roles for the 
users of their servlce&r: a reference service, an extension of SEA staff, 
and a capacity builder. 

• TAC staff act as a reference service for virtually all their 
clients, state and local. This Is the least demanding role. In 
terms of time and skills, but It meets an Important need for 
Information. Whether the Information Is a siammary of the latest 
research findings on effective Instruction or descriptive data on 
available tests, both state and local clients value answers that 
are a telephone call away. 

• Because the /^yerall numbers of SEA stc^ff who specialize In 
Chapter 1 evaluation or program content are so low (averaging 
about half an FTE In each area per state), the PTEs available from 
the TACs make a real difference. SEA staff recognize this, and 
they call on th'*. TACs to fill In for them as providers of help to 
districts, as wexl as to develop materials and procedures they can 
disseminate. 

ft With local staff, TACs have been able to act as capacity builders 
when they and the district commit a reasonable amount of time to 
an Intervention such as a workshop series. After participating In 
a workshop series, for example, school and district staff may be 
able to use evaluation data on a regular basis as they plan and 
monitor program Improvements. At the same time, the TACs can help 
build capacity simply by virtue ot serving as advocates for 
Improvement- -alerting local Chapter 1 coordinators and principals 
to the possibilities of change and Improvement In what may be a 
stodgy, unexamined program. 
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TACs have doxi\, somewhat less work in building SEA capacity, 
althougli our data suggest that this would be a worthwhile 
endeavor. elaborate on this subject, as well as our other 
findings on the effectiveness of TAG services, in the next 
chapters of this report. 
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4. Effects of lAC Services 



This chapter examines evidence on the effectiveness of TAC services » 
based on SEA and LEA reports from our Interviews and the assessments voiced 
by TAC staff. The dlscixsslon begins with an analysis of users' satisfaction 
with the TACs and then analyzes the results of TAC services* 

Users' Satisfaction with the TACs- 

Technical assistance Is delivered by people to people » so the personal 
relationships established between TAC staff and their clients are Important 
Influences on the effectiveness of the services they provide. In fact^ TAC 
staff find that they are in a much better position than federal or state 
assistance givers because they are not identified with compliance monitoring 
or program judgments. The fact that SEA and LEA staff view the TACs as a 
trusted source of information is an important component of their ability to 
be effective. 

Ve were struck by the glowing comments offered by SEA Chapter 1 
coordinators and evaluators in our telephone interviews. Noting the high 
number of projects identified by the Secretary's Recognition Program and 
significant achievement gains of their projects » one SEA director said> 
"These successful gains are a result of the inservice received from the TAC 
and the hands*on relationship with our clients." Other comments » each from 
a different state » included: 

"The [TAC] assistance is always exemplary and supportive." 

"All TAC assistance is useful. The state could not do without their 
services and could benefit from increased services." 
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••Our TAG is trustworthy and does quality work. Everything tuc TAG has 
been doing is significantly helpful.** 

••[TAG is] highly qualified and d<»es a great Job.** 

Effects of the TACs^ Assistance In Evaluation 

TAG and SEA staff report that TAG assistance has resulted in the 
successful adoption and inplementatlon of TIERS. Because assisting in the 
implementation of TIERS was the TAGs' initial purpose, this result is an 
important success for the TAG program* However, TAG staff believe that the 
quality of TIERS data would decline significantly if the TAGs disappeared. 
In the' opinion of TAG staff, SEAs would continue to require LEAs to report 
TIERS data, but they would not exercise^ much leadership in the process nor 
would they provide technical assistance or monitor data quality. 

Responses from state Ghapter 1 staff suggest that the quality of TIERS 
data continues to vary across states, depending upon the degree of 
automation of the SEA's system, the relationship between the SEA and its 
LEAs, and the rate turnover of state and local Ghapter 1 staff. Even so, 
both SEAs and the TAGs themselves agree thnt TAGs have helped improve the 
quality of state*level data. They have done so by helping to implement or 
upgrade qtiality control procedures, including automated systems, that SEAs 
use in entering and editing LEA data. TAG assistance has also significantly 
strengthened procedures for analyzing state -level data and reporting them to 
ED. 

The effects of TAG assistance on the quality of the data that LEAs 
submit to their SEAs are less dramatic, largely because of the sii^^er number 
of LEAs. As an SEA Ghapter 1 coordinator in a small state said, **?.here are 
not enough TAG resources to provide services to every district on a regular 
basis •*• While LEAs are now much more careful about meeting the technical 
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requirements of TIERS (e.g., testing near the norming dates, us. , different 
test results for U:apter 1 student selection and TIERS pri^tests), other 
nethods for improving the quality of district-level TIERS data (e.g.« 
matching tests to curriculc; administering tests properly, recording data 
accurately) require more TAG contact with LEAs than is currently feasible. 
Because TACs are often their only source of Chapter 1 evaluation expertise, 
small districts have probably derived more benefit from TAG help than larger 
districts that employ evaltiators. 

In spite of improvements in data quality, TACs expressed some 
skepticism about: the value of TIERS data beyond its political role in 
Washington. They noted that TIERS results have remained stable over the 
years and thus contain little new information on program effectiveness. 
Moreover, when data are subjected to careful longitudinal analyses, 
short* term gains tend to vanish. Despite this skepticism^ i?ACs view the 
political role of TIERS as critically important in preserving Chapter 1, 
mainly becaiise TIERS has performed a national accountability function for 
the program. TACs also believe strongly in the importance of good 
evaluation data as a starting point for program improvement. In fact, TAG 
staff believe that their assistance in improving evaluation data has made 
the move to program improvement feasible. 

Effects of TACs ^Assistance in ProyraiL Improvement 

Although TAG staff do not systematically track the effects of their 
program improvement efforts, they believe that their assistance in this area 
has had a positive impact. They noted, however, that it is much harder to 
detect effects in program improvement than in evaluation. When TAG staff 
help an SEA prepare an evaluation report or automate a data system, they can 

53 

ERiC ^7 



SM the results. It take&* m\ch longer to discern the results of program 
Inprovement assistance because the intended changes are much more deeply 
embedded in the educational process. Even so, obtained several examples 
of the effects of TAG assistance in program inrprovement. 
Increased Av i ireneas of Relevant Research Findings 

The TACs' dissemination of research on effective schools and classrooms 
is a useful component of TAG assistance in two respects. First* it is a 
valuable service by itself » especially for smaller or more isolated 
districts whose staff are sometimes \maware of alternatives to current 
Chapter 1 instructional arrangements, and SEkn that serve such districts. 
Even in other SFAs and in large LEAs, Chaptei: 1 Ci. rdinatcrs com^ider their 
increased knowledge of research to\ be an impcrtatit resitlt of TAG assista!.:"^ u 
because it helps them perform tlxeir jobs better. Second, the TACs' 
familiarity with recent research serves to legitimize them in the eyes of, 
assistance users and thus helps pave the '^ny for other c^slstance. Onfi, 
Ghapter 1 state coordinator said that tl ^ TAC^s ''assistance is current and 
reliable; [TAG staff] synthesize a lot of information and are up to date on 
the research, which is reflected In the high quality of their asslsnance. 
:A local Ghapter 1 coordinator said that the best feature of the TAC'2 
assistance is ''the expertise that the consultants have and their use 
available rese/^xch in reading and evaluatioh." 

Increasad Use of Evaluation Results 

TAG staff reported that program improvement issistance has increased 
local interest in evaltiation issues because the assistance draws direct 
connections between evaluation results and Chapter 1 programs. As one state 
Chapter 1 coordinator reported, the process of designing program improvement 
^strategies highlights, in particular, the possible uses of evaluation data 
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on sustained effects » since these data indicate where improvement is needed 
most* Becrnse the TACs employ evaluation as the fir?%t step in designing 
improvement strategies » assistance recipients can readily see the link 
between evaluation and improvement* 

One LEA respondent reported that, as a result of TAG assistance » "our 
new approach is that all teachers must know how and plan to meet students' 
needs [using evaluation results]. We have learned how to take evaluation 
data and plot [them] in a meaningful way, so that people can understand 
[them]*" 

Greater Coordination between Chapter 1 and Other Staff 

TAG staff said that one positive effect of the workshops is simply the 
fact^ that they bring together people who have responsibility for Chapter 1 
sesrvices in a school or LEA* TAC staff said some local Chapter 1 
coordinators have not met the principals of some of their districts' 
Chapter 1 schools who, in turn, have not assumed any responsibility for the 
quality of the Chapter 1 program* Similarly, Chapter 1 and regular teachers 
may not have worked together. 

These experiences are particularly valuable given the importance TACs 
(and others) place on improving instructional coordination between Chapter 1 
and regular classroom instruction* Several Chapter 1 coordinators whom we 
interviewed said that their program improvement efforts revealed how 
important it was both to integrate Chapter 1 and regular instruction and 
also to improve the regular school program* 

In addition to the value of these interactions, regional workshop 
participants also appreciate the opportunity to meet and learn from their 
counterparts in other districts, according to TAC staff* 
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Improved Student Performance 

Specific examples of student effects were difficult to obtain , due to 
the length of time required for project changes to result in higher 
achievement scores. Even so, TAG staff cited a few examples of improvements 
in achievement. One district totally overhauled its Chapter 1 program 
design as result of the TAG workshop series and later reported increases in 
student achievement, which It attributed to TAG help. An SEA conducted 
improvement training using the TAG model and reported that the recipient 
district showed achievement gains as a result. 

Our calls to LEAs produced two reports of improved student performance: 

• As a result of TAG help, Chapter 1 staff now see **where time on 
\task increased and the changes [resulted in] student growth,** 

• Due to TAG assistance, "there has been positive growth in three of 
four schools y based on a look at achievement gains prior to the 
project and after the project year; There are also achievement 
gains in comparing project students with similar students,** 

General SEA and LEA Responses to Improvement Assistance 

LEAs tend to be more receptive than SEAs to changes intended to promote 

program improvement. According to TAG staff, LEA feedback during and after 

the workshops is very positive, with some districL^i asking to participate in 

more workshops, TAG staff see growing support from SEAs, which they 

attribute to interest sparked by the nationwide educational reform movement, 

the Secretary's Recognition Program, and satisfied LEA participants in the 

TAG workshops and consultations. This support is visible in several 

developments. Some SEAs now want to specify the number of improvement 

workshops in their letters of agreement. One state has contracted to pay 

for 12 improvement workshops over and above those provided under the TAG 

contract , 
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In addition to support for the TACs' assistance in program improvement, 
SEA respondents give high ratings to the TAG improvement- related products 
with which they are familiar. These include workshop materials, research 
syntheses on characteristics of effective programs, and the "In Their Own 
Words" series. 

Across all our SEA respondents, the main complaint about TAG services 
is that they do not get as much as they would like. As one SEA respondent 
said, ''TAG service is excellent and heeded; we could use all the help we can 
get.'' 

Only one SEA expressed displeasure with xAC assistance in program 
improvement. This SEA found the evaluation assistance to be extremely 
useful but implied that the improvement assistance was too simplistic. The 
state Chapter 1 coordinator said, "I'm not too happy with the people from 
the TAG. New ve refuse to have one person come here. We need [program 
improvement] assistance from people who can really understand districts and 
vadjust to their needs, not jtist give a canned response." After interviews 
with the TAG, we believe that this view reflects (1) a high level of program 
sophistication on the part of this SEA respondent and (2) a TAG staff member 
who is not skilled in the delivery of program improvement assistance. 

Other Effects of lAC Assistance 

SEAs recognize the role that TAGs play in communicating ED priorities 
in program improvement and evaluation. One SEA coordinator said that the 
TAG serves as a liaison between the SEA and ED and between the SEA and the 
state ^s LEAs; he said this is a valuable service and that he would not uant 
to lose it. Another SEA Ghapter 1 coordinator referred to the TAG as "an 
intermediary between the state and ED,** which should be maintained. 
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Conclusion 

Despite variations among TACs, we find that the TACs in general are 
achieving results conunensurate with their efforts in evaluation and program 
improvement assistance. The '^^ost important of these results may be the 
interest and enthusiasm they generate among local Chapter 1 persoxmel. 
Interactions- with TAG staff seem to help LEA staff see new possibilities in 
the instructional services they provide and in the results that their 
student are capable oIl achieving* 

Given the very large nximber of Ghapter 1 LEAs and the small size of the 
TAG program, the positive effects of TAG services will be limited to only a 
few LEA recipients trnless SEAs can be enlisted to participate more 
meaningfully in program improvement assistance* SEA Ghapter 1 offices are 
the logical entities for this work because they know the Chapter 1 program 
and the characteristic of the projects in their states. Iftiat they may lack 
is expertise in program improvement and technical assistance. To help them 
acquira it, they need to learn from the TACs and the TACs need to push them 
to learn. This role would not preclude current TAG services, but it would 
cause TACs and SEAs to place more emphasis on TAG efforts to build SEA 
capacity as assistance providers and advocates for improvement. 
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5. The TACs and 



In this chapter we examine ED's relationship with the TACs from several 
perspectives, including (1) the events and trends that have determined 
federal expectations for the TACs, (2) the administrative decisions and 
procedures that are instrumental in implementing the TAC program, and (3) 
the special circumstaxices affecting technical assistance to Chapter I's 
state-operated programs for migrant students and neglected or delinquent 
students. The chapter concludes by identifying several key results of ED's 
administrative directions to the TACs* 
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Impact of the Larger Chapter 1 Program on the TACs 

Because the TACs exist to advance broad goals of accoxmtability and 
effectiveness in Chapter 1, actions and decisions in other parts of the 
Chapter 1 program influence the TACs. The most important of these are 
reviewed below. 

Changes In Chapter I's Statutory Provisions 

Although OE established the TACs in response to statutory directions in 
1974, ECIA included no similar authorization. Yet, even though the 1981 law 
dropped the statutory requirements for the TACs and the evaluation models, 
ED personnel continued to (1) urge states to collect and report Chapter 1 
evaluation data using the models and (2) provide support through the TACs to 
help in these activities. These policies foxmd support among influential 
groups such as the Chapter 1 state coordinators, who agreed with ED that the 
new law would not reduce congressional interest in monitoring Chapter I's 
effectiveness (Reisner & Harks, 1987). 
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Despite these efforts to ensure continuity, however, the absence of a 
legislative requirement to implement uniform evaluation models has affected 
the work of the TACs. Most obviously, the shift resulted in a drop in the 
number of states submitting comparable Chapter 1 achievement data (from a 
hiijh of 53 SEAs in 1980 to 46 SEAs most recently) . On the other hand, a 
more subtle--and generally positive--change is that, without the legal 
requirement to implement TIERS, ED had to appeal for voluntary 
implementation on the basis of the system's value to SEAs and LEAs. Our 
analysis indicates that this shift helped in some places to change Chapter 1 
evaluation from a compliance responsibility to ah activity conducted mainly 
to generate program information for state and local purposes. 

A second legislative change affecting the TACs was the reduction in the 
Chapter 1 set-aside for SEA administration* ECIA reduced the set-aside from 
the equivalent of 1.5 percent of a state's title I allocation to 1.0 
percent. According to Farrar and Millsap (1986), many SEAs responded to the 
reduction by reassigning staff who had been responsible for Chapter 1 
evaluation and instructional issues. Typical rationales were that (1) 
assistance was available in these areas from the TACs, so that SEA- level 
expertise was not essential, and (2) less Chapter 1 evaluation would be 
needed without the legislative requirement for TIERS. Not surprisingly, 
these decisions increased thvi demands on TAC staff. 
Changes in Ch apter 1 Spending 

Federal appropriations for Title I and Chapter 1 have zigged and zagged 
in response to various policies of cutback and expansion in federal 
education spending, but TAC spending has not paralleled these changes in 
overall Title I/Chapter 1 funding levels. In the first six years of TAC 
operations, TAC funding grew faster than the overall growth in Title I 
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funding, as seen Iti Table 3. Since enactment of ECIA, however, TAG funding 
has declined as a percentage of overall Chapter 1 appropriations, despite 
fluctuations In the total appropriations level. 



Table 3 

ED Spending on Compensatory Education and the TACs, 
1976-77 through 1987-88 



School Year ■ 


TAG Funds 


Total Title 1/ 
Chanter 1 Funds 


TAG Funds as 
Percent of Total 


1976-77 


$1,590,845 


$2,050,000,000 


0.08 


1977-78 


4,087,644 


2,285,000,000 


0.18 


1978-79 


5,297,119 


2,735,000,000 


0.19 


1979t80 


6,648,639 


3 i 228, 382, 000 


0.21 


1980-81 


8.508,050 


3,215,343,000 


6.26 


1981-82 


8,260,388 


3,104,317,000 


0.27 


1982-83 


5,286,232 


3,033,969,000 


0.17 


1983-84 


5,596,979 


3,200,394,000 


6.17 


,1984-85 


4,170,000 


3,480,000,000 


0.12 


1985-86 


3,625,000 


3,688,163,000 


0.10 


1986-87 


3,600,000 


3,529,572*000 


0.16 


1987-88 


3,600,000 


3,944,163,000 


0.09 



Sources: Interviews with ED Budget Service; Relsner Marks, 1987 • 



According to ED analysts Interviewed for this study, the decline in TAC 
funding after 1981-82, relative to overall Chapter 1 funding, reflects a 
belief that the TACs have achieved their main purpose, which was to help 
SEAs and LEAs implement Chapter 1 evaluation systems. The relative decline 
also reflects the demands of other Chap*-er 1 evaluation activities. The 
most Important of these in recent years have been the studies conducted for 
the National Assessment of Chapter 1, mandated in the 1983 technical 
amendments to Chapter 1. 



ERIC 



Tn addition, federal fiscal actions In education have affected the TACs 
In another area- -ED staffing of the Chapter 1 program office and the 
Department's evaliiatlon office. As a result of personnel policies 
implemented in 1981, the office with primary responsibility for 
administering Chapter 1, the Compensatory Education Programs (CEP) office, 
experienced large reductions in total staffing, dropping from 95 staff 
members in April 1981 to 51 in December 1986 (Funkhouser, Mlchie, £e Moore, 
1987) • Reductions were especially large among staff assigned to provide 
technical assistance to SEAs and LEAs; according to Funkhouser et al., ED 
eliminated Chapter 1 specialist positions in parent involvement, needs 
assessment, b»^lc skills, school selection, and neglected or delinquent 
services. During the period one new position was added to work on program 
improvement issues. The net effect was to cut back CEP's activities in 
areas relevant to the work of the TACs. 

Comparable figures are not available on staff members assigned to 
Chapter 1 activities in ED's Planning and Evaluation Service (PES), the 
office that administers the TAC contracts. Respondents in that office 
estimate, however, that the staff assigned to TAC-reiated activities dropped 
from 3.0 FTE staff in 1978 to 0.2 FTE in 1987. 

Cutbacks in the two offices have meant that less staff time is 
available for working with the TACs and for dealing with technical 
assistance issues generally. This is most obvious in reduced staff 
resources available for monitoring the TACs' activities. A related effect 
is the lack of personnel to communicate ED's expectations in Chap ^r 1 
evaluation and program improvement to the states. According to the state 
Chapter 1 coordinators we interviewed, this deficiency has meant that SEAs 
generally rely on the '"ACs for information on ED policies in these areas. 

62 

"ye 



Because the TACs are often called on to explain ED initiatives and 
directions, TAC directors say that it is eissential for them to know what ED 
expects from educational agencies in evaluation and program improvement. 

Growing Importance of Educational Quality Issues in Chapter 1 

TT^.e value of good communication between ED and the TACs has been shown 
in the implementation of the Chapter 1 program improvement initiative. Due 
to a shortage of Chapter 1 discretionary funds and staff resources, ED 
called on the TACs to help publicize and implement the initiative, 
particularly the Secretary's Recognition Program for Chapter 1 projects. 
According to CEP staff, TAC help was crucial in spreading word of the 
initiative among their constitzuencie^ and in helping inform SEAs and LEAs 
about the Recognition Program's selection criteria and its application and 
selection process. 

The TACs welcomed this new responsibility. It came along just as SEAs 
and LEAs were mastering the TIERS requirements and as ED was directing the 
TACs to assist educational agencies in using evaluation data for program 
improvement. The federal initiative helped focus state and local attention 
on the new TAC jiission and, according to one TAC director, prompted some 
projects to begin thinking about how to use evaluation results to improve 
Chapter 1 instruction. 

The results of ED's educational quality emphasis in Chapter 1 are seen 
in reports on the allocation of TAC resources. In the year ended September 
30, 1987, the TACs reported that 55 percent of their staff time devoted to 
direct services went to program improvement activities, as contrasted with 
evaluation- related services, which required 45 percent of their time. This 
allocation is consistent with the requirement in current TAC contracts that 
TACs **reserve approximately 50 percent of their professional time for 
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program improvement activities** (p. 40 of the 1985 Request for Proposals 
[RFP]), 

Continuing Importance of Compliance as Chapter l^s Main Administrative 

The program improvement initiative has not, however, altered ED's chief 
administrative priority under Chapter 1. According to Funkhouser et al. 
(1987), almost all of CEP's staff positions are assigned to compliance- 
related activities, Including policy interpretation and development 
activities (e.g., preparation of guidelines and directives):, compliance 
monitoring, and grants administration. Tlie staff reductions of the 1980s 
did not change these responsibilities but only reduced the number of 
employees available to implement them. 

From the TACs' point of view, CEP's compliance focus has meant that the 
TACs are the only Chapter 1 assistance source for SEAs and LEAs needing help 
in instructional quality or evaluation. Of the SEA and LEA coordinators 
interviewed for this study, only one reported having ever obtained any 
Chapter 1 technical assistance in evaluation or program improvement from ED. 
Indeed, as one TAC staffer said, educational agencies would not take an 
instruction- or evalivition-related problem to CEP (even if they thought CEP 
possessed the relevant expertise) because they would be afraid their inquiry 
might prompt a compliance review. 

Impact of the Procfiures Used To Administer the TACs 

The specific policies and procedures used in administering the TACs 
also contribute to the relationship between the TACs and ED. The most 
important of these a/e discussed below. 
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keliance on the Contracting Process 

Since initiation of the TAG program, OE and then ED have used 
competitive contracts as the basis for administering the TACs. Given the 
limits on federal hiring, federal administrative rules, and the popularity 
of the TACs among tisers, no one seriously suggests that the alternatives to 
thi5f funding vehicle (e.g., creation of a pool of federal consultants to 
provide technical assistancs, special fxmding for states to procure 
assistance directly, use of a le^s confining vehicle such as grants) are 
preferable to contracts. Even so, the use of contracts has several negative 
effects, as described below. 

Priority oh competition . While desirable in most respects, the 
competitive process creates two problems. First, the process itself (e.g., 
developing an RFP, reviewing proposals, convening selection panels, 
negotiating contracts) is so time-consuming that it uses a very large 
portion of total ED staff resources available for administering the TACs. 
As a result, ED staff have little time left for monitoring TAC operations or 
for contact with TAC users. A second problem that competition creates is 
that, according to at least one TAC director, the knowledge that they may 
soon be competing against one another for a TAC contract tends to discourage 
collaboration among TAC offices, including collaboration between the TAC 
prime contractors and subcontractors. 

Excessive federal control over TAC hiring , A key benefit of using 
contracts to operate the TACs is that they allow ED to purchase exactly the 
services and personnel that are desired and no more. Because the contract 
selection and monitoring processes impose no specific limits on federal 
control, however, federal oversight can go too far, as several TAC directors 
described in connection with TAC hiring. One director said that his 
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organization's recruitment activities had been hampered because of 
unreasonable salary limits PES imposes on new hires. Another TAC director 
complained that PES exerts pressure to fill TAC vacancies with evaluation 
specialists, while CEP' expects the TACs to hire educational improvement 
experts. This director felt it was not possible to please both offices. 
These directors said that ED should set broad standards for TAC staff 
qualifications but should delegate recruitment and hiring 'decisions to the 
TAC directors. 

A second area in which ED enerts substantial and perhaps excessive 
control is in TAC activities in'cended to improve the Centers' capacity to 
deliver technical assistance. Current TAC contracts require ED's advance 
approval for the following TiiC activities (p. 42 of the 1985 PJFP) among 
others: 

• Developing ^^compilations of information on successful projects and 
strategies or literature reviews.^ 

• ^Serving on committees to review materials produced, advise on 
evaluation designs, or provide expert opinion on topicr related to 
specific new initiatives.** 

• "Developing materials that can be used across TACs,** such as 
computer software, workshop materials, and handbooks. 

Although ED's advance approval helps ensure that TACs do not duplicate 

each other's efforts in these areas, the requirement for approval deters 

TACs from conducting these capacity building activities. By simply 

requiring TACs to coordinate with each other on these activities, ED could 

minimize duplication of effort, while also encouraging greater collaboratio.. 

across TACs. (On the other hand, by shifting ED's approval to the review of 

TAC products, ED would encourage rigorous quality control, rather than just 

serving as a gatekeeper.) 
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Scheduling rigidity . TAG contracts are awarded for one year plus two 
option years. ED revlevs each TAC's performance before exercising its 
options to continue th;^ contract for the second and third years. We found 
this arrangement to be reasonable except when the contracting cycle is not 
consistent with the timing of Chapter 1 policy changes, as will soon occur. 
Because the current TAG contracts expire in September 1988, ED is currently 
preparing to select new TAG contractors to start work in October 1988. To 
do so, the Department had to prepare specifications for its RFP before the 
Gongress (I) resolved differences in the elementary and secondary 
reauthorization bills passed in the two Houses and (2) voted appropriations 
for the resulting legislation. Because the law contains important changes 
affecting the TAGs (e.Q. , local requirements to identify and improve 
ineffective project services, an ED requirement to fund Chapter 1 rural 
assistance centers), ED may find that its next TAG co?^ tracts do not address 
the needs that new Chapter 1 provisions and regulations will generate. 
Because there is so little flexibility in the contracting process, however, 
ED was forced to adhere to a schedule that required TAG design decisions to 
be made before the new law was passed and appropriations approved. 

Technical Control bv the Evaluation Office 

As indicated above, CEP and PES differ on certain aspects of TAG 
administration, most notably how TAGs should be staffed. Nevertheless, 
respondents^for this study indf ;ated general satisfaction with che current 
administrative arrangement, in which PES provides technical direction for 
the TAG contracts and CEP (1) maintains contact with the TACs (e.g., by 
including the TAGs in all general mailings to SEAs and LEAs and attending 
the TAG directors' meetings in Washington), (2) participates in the 
selection o£ TAG contractors, and (3) stays in contact with PES on matters 




afftcting the TACs. ;>enlor CEP sta£f would like co have a written agreement: 



with PES spellit**' 



-^ach office's responslbllltxes and prerogatives In 



administering t 



PES expressed no Interest in such an agreement. 



The first TAG e\iiluatlon (Mlllman et al., 1979) recommended that OE 
require the TACs to Implement a uniform system of cost and effort reporting. 
That recommendation was accepted, and TAG c6i>tracts since then have Included 
requirements for reporting on TAG activities and expenditures. 

In this evaluation we reviewed several data reporting formats used by 
the TAGs, as followi^: 

• Record of TAG hours and expenditures by contract task 

This format requires monthly reports of TAG effort (in houis per 
named staff maimer) and spending in each of the eight tasks 
Included in the TAG contracts. It thus permits analyses of labor 
and other e: lendltures in the context of specific TAG activities. 
Because it is a standard reporting requirement for federal 
contractors, the organizations operating TAG contracts use it 
frequently and Incur little expense in maintaining it. This was 
by far the most useful quantitative format analyzed for this 
evaluation. 

Its main drawback is that the current tasks mix very broad 
activities, where effort and expense are high (e.g., technical 
consulting in program Improvement), with narrow activities 
requiring little effort or expense (e.g., TAG directors' 
meetings) . 

m Record of topics on which technical assistance is provided 

This format breaks down the effort expended tinder Task 2 of the 
TAG contracts (technical consulting services) into subcategories 
such as data quality control, microcomputer/technology , and 
testing issues. Because actual services almost always span 
several categories, TAG personnel must make arbitrary 
classifications or report "Activities under more than one heading. 

• Record of TAG contact hours 

This format coimts the total hours of TAG service that SEA and LEA 
personnel receive through workshops, on-site consultations, 
telephone calls, and letters. Thus, a 30-minute presentation to 
40 Chapter 1 coordinators at a state Chapter 1 conference yields 
20 contact hours, as do t**^ five-hour days working with one 
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, district's Chapter 1 coordinator and assistant super intehdenc for 

instruction. The chief problem with this fomat is that a contact 
hour is net a particularly useful measure of TAG service as 
discussed in the second chapter of this report, since in-depth 
consultations or workshops with a few decisionmakers are more 
likely to prodixce lasting improvements in Chapter 1 services than 
are briefer, more superficial contacts with large groups. 

• Narrative ''rscriptions of TAC activities 

TACs submit both qxiarterly and annual reports of their activities, 
Including a separate quarterly report devoted to program 
s improvement. The content of thesfe reports varies across TACs, 

though all include both quantitative and narrative reports. In 
bur view the mo^t useful data presented iii the reports are the 
state suiaomaries^ which are ustially presented only in the anntial 
reports. Because TAC .^^ryices are so closely tied to the r.pecial 
circumstances and neeov jf individual states, it is much more 
valuable for an ED monitor to see how TAC services relate to the 
needs and characteristics of a particular state tlian to examine an 
overview of TAC services provided across several states. 

Current TAC narrative reports also include information that is hot: 
useful at all, such as overviews of the history and mission of the 
TAC program or of Title I and Chapter 1. 

Our review of TAC reports helped confirm what we learned from the TACs 

> and from state and local respondents about the mix of (1) direct services to 

SEAs and LEAs and (2) TAC capacity building activities, such as. materials 

development and training of TAC staff. The current TAC focus on direct 

services seems to leave very little room for building internal capacity. 

; All of the requirements and incentives built into the TAC program encourage 

the services focus, from the requirement for ED permission to develop 

materials (as stated in current contracts) to the reporting formats that ask 

only for data on the content and delivery system for direct services. This 

policy is understandable, given the small size of current TAC budgets and 

the h:lgh demand for TAC service, but in general it forces the TACs to rely 

I onwexpertise and materials developed elset^ere. In particular, it may 

xreate special problems as TACs begin to address complex assessment and 

impifovement problems, such as those likely to be prompted by legislative 




provisions requiring LEAs to idencify and assise ineffective Chapter 1 
schools • 

The Special Circumstances Posed for the TACs bv Chapter I's State-Operated 
Programs 

Although not spelled out in the current TAC contracts, one expectation 
EO holds for TACs is that they will provide evaluation and program 
improvement services needed by Chapter 1 migrant education and neglected or 
delinquent projects. Operated directly by SEAs, these two progr:ua areas 
generate demands for technical assistance services that differ substantially 
from those arising in connection with Chapter 1 basic grants. Because of 
these differences, the fit between the services these two programs need and 
those that TACs typically provide is poor. 

In evaluationi for example, the centerpiece of TAC assistance is TIERS, 
which relies on standardized^ tests of Chapter 1 participant, at 12 -month 
incervals. TIERS is difficult to implement in migrant education becatise 
some students move in and out of projects on schedules that do not conform 
to local testing cycles ^ Morecvery migrant students may require tests in 
languages other than English (such as Portuguese and Haitian-Creole), for 
which national nonns may not be available. In neglected or delinquent 
education, students are also likely to enter and exit Chapter 1 projects 
without much notice, precluding pre- and pojt- testing. 

Because of the ^rnique characteristics of projects in these :.reas, 
program improvement strategies that are effective in regular LEA projects 
(e.g., improving the continuity of skills development activities across 
grades, providing follow-up services to students who have "graduated" from 
Chapter 1} may be impossible. 
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In addition to these problems, organizational Issues often separate 
m ^rant and neglected or delinquent services from Chapter 1 basic grant 
services. For example, ED's Migrant Education Program office is separate 
from CEP. States with large migrant populations also sometimes administer 
migrant education in. different offices from the Chapter 1 basic grant 
program. Although the federal Chapter 1 neglected or delinquent program is 
administered in CEP, at the state level the program is usually administered 
in the department of corrections. As n result of these organizational 
arrangements, Chapter 1 migrant education and neglected or delinquent 
programs often have poor links to the basic grant program and thus to thc^ 
main tisers of TAC services. 

Although TACs occasionally work with Chapter 1 migrant programs and 
more infrequently work with neglected or delinquent programs, the TACs are 
not generally well equipped to provide services to these projects. Given 
their special needs and organizational arrangements, it seems unlikely that 
the TACs will be able to devote much attention to these areas, unless ED 
specially designates thern as priorities- -either for all the TAC3 or for one 
or two TACs assigned to maintain the special necessary expertise* 

Results of EDfs Admtntstrative Direction of the TACs 

The preceding discussion higlilights several effects of the current 

relationship between ED and the TACs. In anticipating changes in the 

demands made on the TAC program, we conclude that three effects are 

particularly relevant: 

• Current TAC requirements and incentives placf; low priority on the 
development of expertise or materials specially tailored to the 
needs of TAC users. 

This priority is conveyed in (1) the TAC contracts themselves, 
which require special permission for mo^t internal capacity 
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building activities, and (2) ^he TAG reportir'" requirements, which 
focus on direct services. As a result, TACs ^nust rely oh 
expertise and materials developed by other, non-Chapter 1 sources. 
Because these sources provide expertise and materials that are 
highly relevant to C3iapter 1, we find this to be a reasonable 
strategy in most circumstances. However, it means that TACs have 
few resources for dealing with special user needs or ED 
priorities. 

Few incentives or oppprttmities exist to encourage information- 
sharing or collaboration across TACs. 

The competition built into the relationship among TACs and the 
lack of specific authorization for collaboration in the TP^ 
contracts discourage the TACs from sharing successful appr yaches 
to service delivery. Because the TAC directors' meetings are 
mainly used for communication between the TACs and ED, they do not 
adequately address the need for sharing. across TACs. 

Current federal monitoring procedures do not encourage TACs to 
engage i^* the kinds of in-depth consultation and stistained 
involvement that are most likely to produce lasting improvement in 
Chapter 1 services. 

Given the complexity and uniqueness ' f most program improvement 
needs and the growing importance of ^hese needs among all those 
that TACs address, the TACs' current reporting requirements are 
inappropriate. Since ED must rely on written reports for its 
monitoring of TAC activities (due to shortages of personnel and 
travel funds),, it should revise current reporting requirements to 
highlight (1) state-by-state discussions of Chapter 1 evaluation 
and program impro\ lent needs, TAC services, results, and 
projected needs ana (2) time and resource expenditures by TAC 
activity category. 
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Preceding chapters have described the operations and effects of the 
TACs^ including their interactions with SEAs, LEAs, and ED. Based on the 
information and analyses presented in those chapters and on analyses of 
needs arising from new legislation, ve have identified several areas ir 
which ED might consider modifying the currenc TAG program, these are 
presented in this chapter. 

The suggested changes are premised on our conclusion that ED should 
continue the TAG program. Becaase any recommendations on TAG funding levels 
must consider how funds would be spent if they were not used for TACs, we 
make no suggestions regarding funding levels for TAG services. 

Possible Changes in the Stru.-:ture of the TAG Program 

The changes described in this section address the mission of the TAGs 

and their organization for the delivery of technical assistance services. 

1. Redirect the TACs towards a mission of increasing SEAs^ capacit y 
to promote and Implement Chapter 1 Improvement . 

As discussed in preceding chapters, TACs play three main roles in 

carving SEAs and LEAs; these include serving as (1) sources for information, 

especially in evaluation ^nd testing, (2) extensions of SEA staff, and (3) 

advocates of improvement in Chapter 1 programs. We suggest that ED push 

TAGs to enlarge their role as improvement advocates and capacity builders in 

Chapter 1 program improvement and evaluation. This role takes advantage of 

skills that TAGs possess (and that SEAs and LEAs perceive TACs to possess) 

and encourages TAGs to use these skills in the most efficient way possible, 

which' is to strengthen the ability of SEAs to promote improvement and 

effecv;iva evaltiation in Chapter 1 programs. 
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Because SEAs learn about TAG resources through their services as 
extienslons of SEA staff and Information sources » ED could allow TACs to 
continue these roles as is needed to build relationships with SEAs and LEAs. 
HoweT)er» the TACs could be directed to reduce the amotint of service they 
provide as extensions ot SEA staff and to increase the more substantive 
services they: provide in strengthening SEA capacities. This change will 
require many SEAs to modify their relationships with TACs hy, for example 
Increasing thair participation in and understanding of the improvement 
strategies that TACs promote through workshops and. consultations* 

2. Assign special areas of responsibilltv to some TACs . 

Discussions in preceding chapters note that the TACs have little 
opportunity to develop either special expertise or tailored materials for 
technical assistance activities due to pressures to deliver direct services 
.to SEAs and LEAs. In general » these pressures have yielded positive 
outcomes » including a relatively high level of TAC service to \isers. The 
focus on direct serv'-ie has meant, however, that the TACs are not 
particularly welx equipped to respond to new priorities arising at the 
national level. 

Based on this suggestion, the next TAC RFP could designate a few 
special priority areas and Allow each bidder to propose an approach and 
budget for providing extra servit 3s in that area, if desired. Extra 
services would consist of (1) developing materials for all the TACs to use 
and (2) providing special expertise and advice to other TACs as they plan 
services in the area. TAC contractors selected to provide services in a 
priority area could receive extra resources for that purpose. 
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Based on analysis of new legislation and other information, we have 
identified the following as possible areas for designation as special 
priorities: 

• Parent involvement in Chapter 1, 

• Coordination of Chapter 1 and regular Instruction » 

• Preschool education, 

• Chapter 1 instruction in higher order skills, 

• Services to migrant students, 

• Services to neglected or delinquent students, and 

• Methods of evaluating and Improving the delivery of technical 



3. Establish minimum levels of effort for TAG offices. 
Based on our own analyses and on observations by CEP respondents, we 
believe that very small TAC offices are inefficient* For example, current 
evidence suggests that each office needs a specialist in data base 
development; smaller offices, however, may not be able to use that person's 
time efficiently. To avoid the creation of very small offices, ED could 
consider requiring at least 4.5 PTE staff per TAC office* (The smallest 
office at present is staffed with 3*1 PTE*) This minimum would be 
particularly important if ED decides to change the number or boundaries of 
the TAC regions because it would prevsnt bidders from proposing field 
offices with very small staffs* 

Alternatively, ED might designate a minimum number of states to be 
served by a TAC >ffice* At present no office serves fewer than four states, 
a level that could be established as a national minimum* 

Possible Changes in the General Responstbilitles of the TACs 

The following s gestions concern the work and methods of the TACs* 



assistance* 
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4. Require TACs to evaluate their technical assistance services and 
to use evaluation data to improve their service capacities. 

TACs do not currently evaluate their own activities. As a result, they 
obtain no systematic feedback on the suitability or effectiveness of their 
services. To remedy this surprising gap, we suggest that ED direct the TACs 
to conduct both formative and svumnative evaluations of their activities. 
The TACs' formative evaluations could be designed to assess the services 
they provide in light of what their clients need. It could be measured on a 
state by state basis, utilizing a continuous log of each state's major 
service needs (both as reported by the state and as perceived by TAC staff) 
and TAG services to the state's SEA and LEAs. As services are provided,, the 
TAC staff could assess them to determine whether they addressed major needs 
or whether they simply assisted the state in meeting its ongoing Chapter 1 
obligations. When the TAC determined that services were too heavily 
weighted towards meeting ongoing obligations, it could redirect the services 
towards activities likely to promote lasting improvement in the state's 
Chapter 1 programs (We xmderstand that such changes would need to be 
worked out with the SEA.) 

TAC staff could conduct stuoomatlve evaluations of their services through 
systematic questions (asked orally or in writing) of service r^.cipients, in 
order to determine whether service recipients applied what they had learned 
from the TAC. With at least a sample of these recipients, TAC staff could 
ask follow-up questions at several points in time (e g. , iiunediately after 
the service, six months later ^ one year later) to identify changes planned 
or made as a result of the TAC service. 

The TAC could then use the results of t^ese evaluation activities to 
modify TAC services, in order to make t^em more effective in promoting 
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Chapter 1 Improvement and increasing the capacity of SEA: to act as 
improvement resources for the ISA'S of their states. 

5. Increase TAG capacity to assist in developing student^ and school - 
level data bases. 

New legislative requirements as well as current SEA demands indicate 
the growing importance of state-vide Chapter 1 data bases that can track the 
participation and performance of Chapter 1 beneficiaries. Requirements to 
assess school*levei Chapter 1 perfomance, for example, will require LEAs to 
disaggregate and regroup project -level data to the school level. In 
addition » some states have already requested and received TAC help in 
designing student-level data baf^es to meet their own needs for tracking the 
participation and performance of low achieving students in Chapter 1 and 
state compensatory education programs; most of these data bases have been 
designed for use on microcomputers. 

Although several TAC offices have already deve'^ ^iped this technical 
assistance capacity » anticipaced growth in demand suggests that (1) this 
capacity be made an explicit part of the TACs' responsibilities in 
evaluation and (2) TACs be encovraged to share the software they develop or 
adapt for this purpose. 

6. Increase TAC capacity to assist in coordinating Chapter 1 
evaluatton and program improvement with state testing and 
improvement initiatives. 

Many states have recently adopted extensive new testing programs and 

improvement initiatives as part of state educational refora programs; other 

states are reviewing such proposals or planning for their implementation in 

the near future. Often, however, the state initiatives are not planned to 

coordinate with either (1) testing programs for evaluating Chapter 1 

services or (2) isproyemehts heeded in Chapter 1 programs. We know that the 

need for such coordination is growing as, for example, SEAs and LEAs become 



concerned about (1) the amount of achievement testing occurring in many 
classrooms and (2) the exclusion of compensatory education programs from the 
implementation of some improvement initiatives. To help meet SEA and LEA 
needs t EO could require TACs to possess the capacity for assisting 
educational agencies in these areas. 

7. Conduct a series of TAG seminars on technical assistance issues . 
As discussed in preceding chapters, TACs have little opportunity or 

incentive to share information and successful approaches among themselves. 
To encourage greater sharing, ED could authorize the TACs to conduct 
periodic seminars on topics of major impor^ance to all the TACs. A TAC with 
particular expertise or interests in the stibject would organize and host 
each seminar, and TAC staff and others (e.g., SEA and LEA experts, 
researchers) could make presentations and lead .>:tissions. Given the 
amount of effort required for each seminar, one per yer.r would be 
sufficient. To ensure that other topics do not crowd out the main purpose 
of the seminisr, we suggest that the seminars be held outside Washington. 
Possible seminar topics include: 

• Reading instruction in Chapter 1 projects, 

• Integrating Chapter 1 testing with state testing programs, and 

• Microcomputer data bases for tracking the participation and 
performance of Chapter 1 participants. 

8. Publish a national newsletter on Chapter 1 program improve ment and 
evaluation . 

Because of the growing importance of evaluation and program improvement 
in proposed legislation and in the states, technical assistance users would 
probably welcome a national newsletter coveri4.j these topics. The 
newsletter could include stories on relevant ED activities, state 
initiatives, research findings, new TAC services, and features on state and 
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local **successes, ** The newsletter could be the responsibility of one of the 
TACs, with all the TACs urged to submit articles* Its publication would be 
particularly timely as SEAs and LEAs prepare to implement new Chapter 1 
legislation* 

Possible Changes in ED Admtrtstration of the TACs 

The following suggestions are concerned with ED's management and 
^monitoring of the TACs. 

9« Implement new reporttny requirements. 

Chapter 5 of this repcrt described the TACs' current reports to ED on 
their activities and services. We conclude that ED could streamline current 
reporting \ j tccosing on (1) monthly reports of TAC expenditure of staff 
time and other resources across TAC activities and (2) state-by-state 
narrative reports of evaluation and program improvement needs , TAC services > 
results > and projected needs. 

10. Increase feedback to the TACs on their performance. 

Because of other demands » ED ataff responsible for monitoring the TACs 
have little opportunity to review performance or to provide constructive 
foedback. ED might consider staff arrangements that would permit ED 
moniturs more time to (1) read and question TAG activity reports, 

(2) consult with users of TAC services to assess TAC performance, and 

(3) provide feedback to the TACs on the strengths and weaknesses of their 
services. These monitoring activities would assist TACs in their self- 
evoltiatioh activities. They would also help ED monitors stay informed about 
evaluation and program improvement issues in Chapter 1. 
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11. Allow TACs greater latitude tn staffing. 

An area in which the TACs may need a bit more slack from ED is in 
staffing decisions. The TACs' current contracts set >at reasonable 
standards for TAC directors, assistant directors, and y Ject staff, which 
were used to assess the qualifications of the individuals proposed to staff 
the current contracts. ED cculd allow TAG directors greater latitude in 
deciding whom to hire as they replace TAC staff members who leave. We find 
this a reasonable delegation of decisionmaking because TAC directors are in 
the befit position to decide whether a job candidate is likely to contribute 
to a TAC's overall capacity for meeting the technical assistance needs of 
its region. The exception might be the positions of TAC director or 
director of a TAC field office, for which ED coula continue to review 
proposed new hires. 
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